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TOWARDS A MATHEMATICAL DEFINITION OF COULOMB 
BRANCHES OF 3-DIMENSIONAL M = 4 GAUGE THEORIES, I 

HIRAKU NAKAJIMA 


Abstract. Consider the 3-dinrensional J\f = 4 supersymmetric gauge theory associated 
with a compact Lie group G and its quaternionic representation M. Physicists study its 
Coulomb branch, which is a noncompact hyper-Kahler manifold, such as instanton moduli 
spaces on R 4 , SU(2)-monopole moduli spaces on R 3 , etc. In this paper and its sequel, we 
propose a mathematical definition of the coordinate ring of the Coulomb branch, using 
the vanishing cycle cohomology group of a certain moduli space for a gauged cx-model on 
the 2-sphere associated with (G, M). In this first part, we check that the cohomology 
group has the correct graded dimensions expected from the monopole formula proposed 
by Cremonesi, Hanany and Zaffaroni [CHZI ]. A ring structure (on the cohomology of a 
modified moduli space) will be introduced in the sequel of this paper. 


1. Introduction 

l(i). 3-dimensional A^ = 4 SUSY gauge theories. Let M be a quaternionic represen¬ 
tation (also called a pseudoreal representation) of a compact Lie group G. Let us consider 
the 4-dimensional gauge theory with AT = 2 supersymmetry associated with (G, M). It 
has been studied by physicists for many years. It is closely related to pure mathematics, 
because the correlation function of its topological twist ought to give the Donaldson in¬ 
variant [Don9U] of 4-manifolds for (G, M) = (SU(2),0), as proposed by Witten [ ]. 

Thus physics and mathematics influence each other in this class of gauge theories. For 
example, Seiberg-Witten’s ansatz [SW94] led to a discovery of a new invariant, namely the 
Seiberg-Witten invariant. It is associated with (G, M) = (U(1),H). Nekrasov partition 
function [Nek03] gave a mathematically rigorous footing on Seiberg-Witten’s ansatz, and 
hence has been studied by both physicists and mathematicians. 

In this paper, we consider the 3-dimensional gauge theory with A^ = 4 supersymmetry, 
obtained from the 4 d theory, by considering on (3-manifold) x and taking R —)■ 0. We 
denote the 3d gauge theory by Hyp(M) -fjf-G following [Tac], though it is used for the 4 d 
gauge theory originally. One usually studies only asymptotically conformal or free theories 
in 4 d, while we do not have such restriction in 3d. Also some aspects are easier, simplified 
and clarified in 3d, and our hope is that to use understanding in 3d to study problems in 
Ad. 

We do not review what is Hyp(M) jff- G. It is a quantum field theory in dimension 3, 
and is not rigorously constructed mathematically. 

Instead we will take the following strategy: physicists associate various mathematical 
objects to Hyp(M)^G and study their properties. They ask mathematicians to construct 
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those objects, instead of Hyp(M) -ffj- G itself, in mathematically rigorous ways so that 
expected properties can be checked. 

This strategy is posed explicitly in [ ] for a particular object, which is very close 

to what we consider here. (See §3(iii) for detail on the precise relation.) Many physically 
oriented mathematical works in recent years are more or less take this strategy anyway. 

1 (ii). Topological twist. An example of mathematical objects is a topological invari¬ 
ant defined so that it ought to be a correlation function of a topologically twisted ver¬ 
sion of Hyp(M) jjf- G. For the 4-dimensional J\f = 2 SUSY pure SU(2)-theory (i.e., 
(G,M) = (SU(2),0)), Witten claimed that the correlation function gives Donaldson in¬ 
variants, as mentioned above. In [ vI90] Atiyah- Jeffrey understood the correlation function 
heuristically as the Euler class of an infinite rank vector bundle of an infinite dimensional 
space with a natural section s in the Mathai-Quillen formalism. The zero set of s is the 
moduli space of SU(2) anti-self-dual connections for (G, M) = (SU(2),0), hence Witten’s 
claim has a natural explanation as a standard result in differential topology applied for¬ 
mally to infinite dimension. It was also observed that the 3-dimensional story explains 
an Taubes’ approach to the Casson invariant [Tau90], at least for homological 3-spheres. 
And 3 and 4-dimensional stories are nicely combined to Floer’s instanton homology group 
[Flo88] and its relation to Donaldson invariants in the framework of a (3 + l)-dimensional 
topological quantum held theory (TQFT) [Don02]. Another example of a similar spirit is 
the Seiberg-Witten invariant in dimensions 3/4 (see e.g., [Wit94] for Ad and [MT96] for 
3d). This is the case (G,M) = (U(1),HI). 

Atiyah-Jeffrey’s discussion is heuristic. In particular, it is not clear how to deal with 
singularities of the zero set of s in general. We also point out that a new difficulty, besides 
singularities of Zero(s), failure of compactness occurs in general. See §l(iv) and §6(ii) 
below. Therefore it is still an open problem to define topological invariants rigorously 
for more general Hyp(M) -ffj- G. It is beyond the scope of this paper. But we will use a 
naive or heuristic analysis of would-be topological invariants to help our understanding of 
Hyp(M) HfG. We hope the study in this paper and its sequel [ UN 16a] might be relevant 
to attack the problem of the definition of topological invariants. 


l(iii). Coulomb branch. Instead of giving definitions of topological invariants, we con¬ 
sider the so-called Coulomb branch M.c of Hyp(M) jffG. 

Physically it is defined as a specific branch of the space of vacua , where the potential 
function takes its minimum. However it is only a classical description, and receives a 
quantum correction due to the integration of massive fields. At the end, there is no 
definition of Aic, which mathematicians could understand in the literature up to now. 
However it is a physicists consensus that the Coulomb branch is a hyper-Kahler manifold 
with an SU(2)-action rotating hyper-Kahler structures /, J, K [ >W97]. Physicists also 
found many hyper-Kahler manifolds as Coulomb branches of various gauge theories, such 
as toric hyper-Kahler manifolds, moduli spaces of monopoles on M 3 , instantons on M 4 and 
ALE spaces, etc. (Reviewed below.) 
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l(iv). A relation to topological invariants. The Coulomb branch M. c might play 
some role in the study of would-be topological invariants for Hyp(M) jfj-G. As topological 
invariants are currently constructed only (G, M) = (SU(2),0) and (G, M) = (U(1),EI), 
we could touch this aspect superficially. Nevertheless we believe that it is a good starting 
point. 

First consider Hyp(O) t^SU( 2). Let us start with the 4-dimensional case. A pseudo 
physical review for mathematicians was given in [ CY04, §1], hence let us directly go to the 
conclusion. The space of vacua is parametrized by a complex parameter u, and hence called 
the u-plane (Seiberg-Witten ansatz). We have a family of elliptic curves E u parametrized 
by u, where E u degenerates to rational curves at u = ±2A 2 . The so-called prepotential of 
the gauge theory is recovered from the period integral of E u . From the gauge theoretic view 
point, u is understood as a ‘regularized’ integration of a certain equivariant differential form 
over the framed moduli space of SU(2)-instantons on M 4 . More rigorously we define the 
integration by a regularization cooperating Fraction on M 4 (Nekrasov’s D-background). 
The prepotential above determines the equivariant variable a as a function of u, and hence 
u as an inverse function of a. 

Witten explained that the Donaldson invariant is given by a u-plane integral, and the 
contribution at the singularities u = ±2A 2 is given by the Seiberg-Witten invariant [ ]. 

See also [ ] for a further development. This picture was mathematically justified, in 

a slightly modified way, for projective surfaces in [ NY08, GNY11]. 

Now we switch to the 3-dimensional case. As is mentioned above, we first consider 
M 3 x and take the limit R —>• 0. Seiberg-Witten determines the space of vacua for large 
R as the total space of the family E u for u G C [SW97]. When we make R —> 0, points 
in E u are removed, and get the moduli space of charge 2 centered monopoles on M 3 . This 
is a 4-dimensional hyper-Kahler manifold studied intensively by Atiyah-Hit chin [AH88]. 
This is the Coulomb branch Aic of the 3-dimensional gauge theory Hyp(0) ^SU(2). It 
means that the 3-dimensional gauge theory Hyp(0) y^SU(2) reduces at low energies to a 
sigma-model whose target is M. c . 

This picture gives us the following consequence for topological invariants. The parti¬ 
tion function for the twisted Hyp(0) jjf- SU(2) is the Casson-Walker-Lescop invariant as 
above. On the other hand, the partition function for the sigma-model with target M. c \s 
the topological invariant constructed by Rozansky-Witten [ \Y97]. Then as an analog of 
Seiberg-Witten = Donaldson in 4-dimension, it is expected that the Casson-Walker-Lescop 
invariant coincides with the Rozansky-Witten invariant. 

The Rozansky-Witten invariant for M. c is a finite type invariant [Oht96, LM098, 
LMM099] of order 3, which is unique up to constant multiple. Therefore the coinci¬ 
dence of two invariants is not a big surprise. But it at least gives an expectation of a 
generalization to G = SU(r). The Coulomb branch of Hyp(0) ^SU(r) is the moduli space 
of charge r centered monopoles on M 3 [CH97]. Neither the SU(r)-Casson-Walker-Lescop 
invariant and the Rozansky-Witten invariant associated with are not mathematically 
rigorously defined yet, but it is natural to expect that they coincide once they would be 
defined. For the former, singularities of moduli spaces must be treated appropriately. See 
[CLM90, BH98, BHK01, CLM02], for example, studying this problem. The convergence 
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of an integral must be proved for Rozansky-Witten invariants, as the monopole moduli 
spaces are noncompact. 

Let us continue examples from [SW97]. Let G = U(l), M = W N = C N © (C*)^, the 
direct sum of N copies of the vector representation plus its dual. The Coulomb branch 
M.c is the multi-Taub-NUT space, which is C 2 /(Z/IVZ) as a complex variety when N > 0, 
and is M 3 x S 1 = C x C* when N — 0. 

The case N — 0 is trivial, so let us exclude it. There is a big distinction between N — 1 
and N > 1 cases, where we have a singularity at the origin or not. For N — 1, the 
partition function gives the 3-dimensional Seiberg-Witten invariant. On the other hand, 
M. c is the Taub-NUT space, the associated Rozansky-Witten invariant is again finite type 
of order 3, hence should be equal to the Casson-Walker-Lescop invariant up to multiple 
again. Marino-Moore [MM99], Blau-Thompson [ ST01] argued that the Casson-Walker- 
Lescop invariant is equal to the 3-dimensional Seiberg-Witten invariant, more precisely, 
the (regularized) sum over all Spin c classes, as proved earlier by Meng-Taubes [MT96] for 
b\ > 0. In the b\ = 0 case, the definition of the Seiberg-Witten invariant is more subtle, 
and the claim was shown later by Marcolli-Wang [MW02], 

For N > 1, the Coulomb branch M. c has a singularity at the origin, whose contribu¬ 
tion to the Rozansky-Witten invariant needs to be clarified. In the Seiberg-Witten side, 
compactness of moduli spaces fails, as we will review in §6(ii). Therefore it is not yet clear 
how to define the invariant. The singularity comes from the Higgs branch M.h, explained 
later. Therefore it is natural to expect that the Rozansky-Witten invariant for M. H enters 
the picture. 

Let us note that M.c for Hyp(0) ^fSU(2) can be defined as a limit of the total space of 
E u , which is rigorously recovered from Nekrasov’s partition function. This method could 
probably apply to if (G , M) gives an asymptotically conformal or free theory in dimension 
4, but not in general. For example, it is not clear how to do for (U(1),HI). 

l(v). (2 + l)-dimensional TQFT. The Rozansky-Witten invariant associated with a 
hyper-Kahler manifold M is expected to fit in the framework of a (2 + l)-dimensional 
TQFT. For a 2-manifold £, one associates a quantum Hilbert space 'Hy. and an invariant 
of a 3-manifold A" with boundary £ takes value in "Hy. Then the gluing axiom is satisfied. 

In [ YV97, §5], it is proposed that 

Hz, = (-1) 1+J © H\M, (AT)»), 

9 

where E g is a 2-manifold of genus g , V is the natural Sp(dinie M)-bundle over M, and the 
sign (—1) 1+9 is introduced so that has a correct Z/2-graded vector space structure. 
Rozansky-Witten wrote that it is hazardous to apply this definition for noncompact M , 
like our Coulomb branch M.c- 

Nonetheless consider the case g = 0 assuming M. c is affine: 

n S 2 = ®(-i ) 1 + m«(M c ,o) = -c [Me]. 

q 
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Let us give a nontrivial check for this hazardous assertion. By the gluing axiom, the 
invariant of S 2 x S 1 is equal to the dimension of 'H-s 2 - O n the other hand, from the 
knowledge of the Casson-Walker-Lescop invariant for S 2 x S 1 , it should be equal to 1/12 if 
Me is the Taub-NUT space or the Atiyah-Hitchin manifold (see [ W97, (4.4), (5.15)] and 
[BT01, (2.3)]. Therefore we should have dim C[A4c] = —1/12. For the Taub-NUT space, 
which is isomorphic to C 2 as an affine variety, this is true after ^-regularization: 


OO 1 

n=l t= x 

The expression 1/(1 — t) 2 is coming from the natural grading on C[A 4c] — C[x, y] as 
deg a: = degy = 1. See §4(ii) for more detail. A closely related observation on —1/12 can 
be found at [MM99, (6.16)], [BT01, §2.3]. It is interesting to look for a similar explanation 
for the Atiyah-Hitchin manifold, as well as a deeper understanding of this regularization 
process. 

On the other hand, the Casson and Seiberg-Witten invariants are also expected to fit 
in the TQFT framework. The quantum Hilbert spaces Hy. are the cohomology group 
of moduli spaces of flat SU(2)-bundles and solutions of the (anti-)vortex equation over 
£ respectively [Ati88, Don99] (see also [Ngul ]). This cannot be literally true for the 
Casson case, as there is no nontrivial flat SU(2)-bundle on £ = S 2 . In the Seiberg-Witten 
case, 3d invariants depend on a choice of perturbation of the equation, and its dependence 
must be understood via the wall-crossing formula. We need to choose the corresponding 
perturbation of the (anti-)vortex equation to have a nonempty moduli space in 2d. For a 
‘positive’ (resp. ‘negative’) perturbation, moduli spaces for the vortex (resp. anti-vortex) 
equation are nonempty, and symmetric products of £. For £ = S 2 , the (n — l)th symmetric 
product S' ri “ 1 £ is P n_1 , and its cohomology is n-dimensional. It is compatible with the 
above naive computation (1.1), as n appears in the middle. 

If Casson-Walker-Lescop = Rozansky-Witten would be true as (2 + l)-TQFT’s, we con¬ 
clude that 


( 1 . 1 ) 


(i -ty 


C[A4c of Hyp(O) ^SU(2)] — H *(moduli spaces of flat bundles on S 2 ), 

and similarly for €.[M.c of Hyp(HI) jjj- U(l)] and the cohomology group of moduli spaces 
of solutions of the (anti-)vortex equation on £ = S 2 . This could not be true for flat 
bundles as we have remarked above. For the (anti-)vortex equation we do not have an 
immediate contradiction, but it is too strong to be true, as the Coulomb branch seems to 
be independent of the choice of perturbation. 


l(vi). Monopole formula. As we have explained just above, it seems difficult to use our 
current understanding of TQFT to determine M.c hi a mathematically rigorous way. 

A work, more tractable to mathematicians, has been done recently by Cremonesi, 
Hanany and Zaffaroni [ ]. They write down a combinatorial expression, which gives 

the Hilbert series of the Coulomb branch A4c- It is called the monopole formula. 
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The monopole formula is a formal Laurent power series 

(1.2) H GM (t) d = ]T t 2A ^P G (t-,X), 

A eY/W 

where Y is the coweight lattice of G, W is the Weyl group, and A(A), A) are certain 
an integer and a rational function in t respectively. We postpone a detailed discussion of 
the monopole formula to §4(i). Let us give a brief comment here: (1.2) is a combinatorial 
expression, and mathematically makes sense contrary to the case of Me¬ 
lt is worthwhile to keep a physical origin of the monopole formula in mind. The monopole 
formula counts monopole operators, which are defined by fields having point singularities 
[BKW02], Taking a radial coordinate system around a singular point, the singularity is 
modeled on a connection on S 2 , whose topological charge is given by a coweight of G. This 
is the reason why the coweight A appears in (1.2). We do not review physical origins of 
expressions A(A), Pcifi A). See [CHZ14, §2] and the references therein. 

It is also clear that monopole operators belong to the quantum Hilbert space Hg 2 f° r S 2 . 
They form a ring (called a chiral ring in physics literature), by considering the topological 
quantum held theory associated with S 3 with three punctures. 

Combining with our heuristic consideration in the TQFT framework, we will take the 
following strategy to find a definition of A4 G . We will start with the cohomology group of 
a moduli space, and look for its modification so that its Poincare polynomial reproduces 

(1.2) . Then we will study properties of the proposed Coulomb branch, whether they are 

compatible with physical expectations. We will propose such modification in this paper 
and its sequel [ ; N16a], 

Let us give a remark. In (1.2) we need to assume 2A(A) > 1 for any A ^ 0, a ‘good’ 
or ‘ugly’ theory in the sense of [ 1W09], hence no negative powers of t appear. Then 

(1.2) makes sense as a formal power series. This assumption fails for example the pure 
theory (G, M) = (SU(2),0), though M. c still exists and is the monopole moduli space as 
we discussed above. Our proposed definition, though motivated by (1.2), will make sense 
without this assumption. 

l(vii). Higgs branch, 3 d mirror symmetry and symplectic duality. There is a con¬ 
struction of a hyper-Kahler manifold from G and M, which mathematicians can under¬ 
stand. It is the hyper-Kahler quotient construction [ ]. (See §2(i) below for detail.) 

However the Coulomb branch JY[ C is not the hyper-Kahler quotient of M by G. In the 
above examples with M = 0, the hyper-Kahler quotient M/j/G is just {0} for any G. (Or 
0 if we consider only free orbits.) For (G, M) = (U(l), M N ), the hyper-Kahler quotient 
M^/G is the closure of the minimal nilpotent orbit in sl(N, C), or the cotangent bundle of 
CpA-i if the real part of the level of the hyper-Kahler moment map is nonzero. In fact, 
the hyper-Kahler quotient of M by G arises as the Higgs branch JY[ H , which is yet another 
mathematical object associated with the gauge theory Hyp(M) -fff-G. 

For this class of 3-dimensional supersymmetric theories, it has been noticed that two 
theories often appear in pairs. It is called the mirror symmetry in 3-dimensional theories, 
as it is similar to more famous mirror symmetry between two Calabi-Yau’s. The first set 
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of examples was found by Intriligator and Seiberg [1S96]. In fact, the above is one of their 
examples, where the mirror theory is the gauge theory for G = 11(1)^/ U(l) with M = M N 
associated with the affine quiver of type (See §2(iv) below.) 

When two theories A, B form a mirror pair, their Higgs and Coulomb branches are 
swapped: 

This is indeed the case for our example. The hyper-Kahler quotient of M. N by 11(1)^/ U(l) 
is C 2 /(Z/iVZ), Kronhcimer’s construction of ALE spaces for type A [Kro89]. 

Remark 1.3. In order to the above equalities to be literally true, we need to replace the 
multi-Taub-NUT spaces by the corresponding ALE spaces. (The multi-Taub-NUT metric 
has a parameter g c \, and it becomes the ALE space when g c \ —> oo.) This is because the 
mirror symmetry is a duality in infrared. See [IS96, §3.1] for a physical explanation why 
this is necessary. As we are only interested in the complex structure of A ic, this process 
makes no change for us. We ignore this point hereafter. 

Therefore the monopole formula (1.2) for the theory A computes the Hilbert series of 
the Higgs branch of the mirror theory B. And there are lots of examples of mirror 
pairs, and we have a systematic explanation via branes, dualities, M-theory, etc. See e.g. 
[dBH0097, PZ97, HW97, dBHO+97] and also §§2(v), §3. 

Unfortunately these techniques have no mathematically rigorous foundation, and hence 
there is no definition of the mirror, which mathematicians can understand. Moreover, the 
mirror of a gauge theory Hyp(M) fff-G may not be of a form Hyp(M / ) jff-G' for some M' 
and G' in general, as we will explain in §3(i). Thus it seems that the mirror symmetry is 
more difficult to work with, and we will not use it to look for the mathematical definition 
of M.q. In turn, we could hope that our proposed definition of M.q will shed some light 
on the nature of the 3d mirror symmetry. 

Let us continue an example of a mirror pair. The pair in Figure 1, both of quiver 
types, is known to be mirror each other [ BH0097]. Higgs branches are the k th symmetric 
power of C 2 /(Z/WZ) and the framed moduli space space of S\J(N) fc-instantons on M 4 , 
given by the ADHM description respectively. This includes the above example as k = 1 
case. The former can be considered as the framed moduli space of U(l) fc-instantons on 
C 2 /(Z/./VZ). Their Hilbert series has been computed. For Ad#, it can be written in terms 
of the Hilbert series of C 2 /(Z/AdZ) as it is a symmetric product. For Af#, the Hilbert 
series is the /l-theoretic Nekrasov partition function (or 5-dimensional partition function 
in physics literature), and can be computed via fixed point localization or the recursion by 
the blowup equation, e.g., see [NY05b]. 

We should emphasize that it is not obvious to see why the monopole formula reproduces 
those results, as (1.2) looks very different from the known expression. For A4q, this is 
possible after some combinatorial tricks. See §B. (It was checked in [ ] for small k.) 

But it is not clear how to check for A Aq. 

Let us mention that Braden et al. [ ] expect that the mirror symmetry is re¬ 

lated to the symplectic duality, which states an equivalence between categories attached 
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Mff : U(l) Auinstantons MS : SU(iV) fc-instantons 
on C 2 /(Z/NZ ) on M 4 

N 

6 

FIGURE 1. An example of a mirror pair 

to symplectic resolutions of two different conical hyper-Kahlcr manifolds Mjj = MS and 
MS = M b h . The definition of categories and the dual pair require also that both sym¬ 
plectic resolutions have torus action with finite fixed points. Since Mjj, MS do not have 
symplectic resolutions nor torus action with finite fixed points in general, the symplectic 
duality deals with much more restrictive situations than ones considered here. If Mfj 
satisfies these two conditions, it is natural to expect the same is true for MS, as twos 
are interchanged under the mirror symmetry, as will be explained in §5(i). Note that it is 
usually easy to check these conditions for Mjj, and we have lots of examples, say quiver 
varieties of type A or affine type A, toric hyper-Kaher manifolds. 

No general recipe to construct a symplectic duality pair was given in [ PW14]. Since 

we will propose a definition of MS in this paper, this defect will be fixed. Moreover we hope 
that we could give a better understanding on the symplectic duality, and gain a possibility 
to generalize it to more general cases when two conditions above are not satisfied. We have 
interesting sets of examples, where only one of two conditions is satisfied. 

l(viii). Hikita conjecture. Recently Hikita [Hikl6] proposes a remarkable conjecture. 
Suppose that G is a product of general linear groups, such as quiver gauge theories §2(iv) 
and abelian cases §2(vi). Using perturbation of the moment map equation, we can modify 
the hyper-Kahler quotient M///G to ii~ 1 (()/G. In many cases, it is a smooth manifold, 
and assume that this happens. 

On the Coulomb branch side, we have an action of a torus T on Me, where T is the 
Pontryagin dual of 7 Ti(G). See (c) in §4(iii) below. Let MS be the fixed point subscheme. 
Hikita conjectures that there exists a ring isomorphism 

C[ M T C }^ H'{v.-\QIG), 

and checks for several nontrivial cases. This conjecture obviously has a similar flavor with 
our study, though a precise relation is not clear yet. The author is currently considering 
what happens if we consider the equivariant quantum cohomology group of /x~ 1 (C)/G. It 
seems the quantized Coulomb branch defined in [ N16a] plays a role. 

l(ix). A proposal of a definition of Me- We take the cohomology group of moduli 
spaces of solutions of the generalized vortex equation for the gauged nonlinear cr-model 
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on S 2 as a starting point for a definition of A4c, as discussed in §l(v). The equation will 
be discussed in detail in §6. See (6.9). However we need to modify the definition, as we 
cannot obtain a reasonable answer for Hyp(O) jff- SU(2), as we have already remarked. 

Our proposal here is the following modifications: 

(1) Drop the last equation in (6.9), which is related to the stability condition via the 
Hitchin-Kobayashi correspondence. 

(2) Consider the cohomology group with coefficients in the sheaf of a vanishing cycle. 
Thus we propose 


C[M c ] = 


*+dim T— dim Qc (P) 


(A,*) 


(d + A)$ = ol 

Mc($)= 0 J 


^cs(Qf))*, 


where d + A is a partial connection on a Gc-bundle P on S 2 = P 1 , $ is a section of an 
associated vector bundle twisted by Opi(— 1), and /l c is the complex moment map. And T 
is the space of all ( A , <f>) imposing no equations, Gc(P) the complex gauge group, and tpcs 
is the vanishing cycle functor associated with the generalized Chern-Simons functional CS 
defined on T. See §7 for more detail. 

The moduli space above can be loosely regarded as the space parametrizing twisted 
holomorphic maps from P 1 to the Higgs branch M.h- It is literally true if we replace 
Mh = Me 1 (0)/G r c by the quotient stack [/x^ 1 (0)/Gc]. 

The vanishing cycle functor <^cs with respect to the generalized Chern-Simons func¬ 
tional CS is strongly motivated by the one appearing in the theory of Donaldson-Thomas 
invariants for Calabi-Yau 3-categories. 

The whole paper is devoted to explain why these modifications are natural. We see 
that (1) is inevitable even at this stage: (6.9) is just Fa — 0 for Hyp(O) ^SU(2). We 
cannot think of any reasonable modification, which gives us a non-trivial solution for S 2 . 
For Hyp(HI) ^U(l), moduli spaces depend on the choice of a stability condition, though 
the Coulomb branch should not. This problem apparently is related to the dependence 
of invariants of perturbation, mentioned at the end of §l(v). It has a similar flavor with 
the problem arising the definition of SU(r) Casson invariants, mentioned in §l(iv). Thus 
forgetting the equation and considering all connections seem the only reasonable candidate 
for the modification. 

The definition of the multiplication, when M is of cotangent type as explained below, 
will be postponed to [ lFN16a]. The goal of [ FN16a] will be to propose a definition of Aic 
as an affine scheme, i.e., the definition of its coordinate ring as a commutative ring. There 
remain lots to be done, in particular, we have no idea how to define a hyper-Kahler metric 
on A4c at this moment, though we could construct a natural noncommutative deformation 
(or quantization) of Me- 

Also we are very far from checking our proposal reproduces various known examples 
already mentioned above, except M = 0 and toric cases. Nevertheless we will reproduce 
the monopole formula (1.2), so we believe that our proposal passes the first check that it 
is a correct mathematical definition of Coulomb branches. 
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The paper is organized as follows. In §2 we review the hyper-Kahler quotient construction 
and examples of hyper-Kahler manifolds arising in this way. In §3 we give examples where 
the mirror of a gauge theory is not a gauge theory though it is still a reasonable theory. In 
§4 we review the monopole formula and its various properties. In particular, we start to list 
expected properties of the Coulomb branch suggested from the monopole formula. In §5 
we review the monopole formula when a gauge theory has an additional flavor symmetry. 
We add a few properties to the list. Up to here, all materials are review of earlier works. 

In §6 we consider a generalized Seiberg-Witten equation associated with a gauge theory 
Hyp(M) jjj- G and study the compactness property of the moduli space. We also study 
the dimension reduction of the equation to write down the generalized vortex equation, 
which will lead us to the proposed definition of the Coulomb branch. In §7 we observe that 
the complex part of the reduced equation on a Riemann surface arises the Euler-Lagrange 
equation of an analog of the holomorphic Chern-Simons functional, and hence it is natural 
to consider the analog of Donaldson-Thomas invariants, or more precisely the cohomology 
of the vanishing cycle. We then formally apply results on the vanishing cycle, known in the 
finite dimensional situation, to our case to reduce the equation further. In §8 we compute 
the dimension of the cohomology group and check that it reproduces the monopole formula. 

§9 is a detour where we find that a few earlier works are nicely fit with the framework of 
§7 when the curve is the complex line C. 

In §A we give further examples of hyper-Kahler quotients related to instantons for clas¬ 
sical groups. In §B we give a computation of the monopolc formula in a particular example 
when the Coulomb branch is a symmetric product of a surface. 
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2. Examples of hyper-Kahler quotients 

In view of the 3d mirror symmetry, it is natural to expect that the Higgs branch A in 
and the Coulomb branch M.q share similar properties. Therefore it is important to have 
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examples of gauge theories whose Higgs branches (i.e., hyper-Kahler quotients of linear 
spaces) are well-understood. In this section we prepare notation and basics of hyper- 
Kahler quotients in the first three subsections, and then we review two important classes 
of gauge theories, quiver gauge theories and abelian theories. Further examples will be 
given in §A. 

Here we consider hyper-Kahler quotients only in finite dimension. If we allow infinite 
dimensional ones, we have more examples, such as instanton moduli spaces for arbitrary 
gauge groups, solutions of Nahm’s equation, etc. However it is not clear how to consider 
the corresponding Coulomb branches, in particular, the monopole formula introduced in 
§4 below. 

2(i). Hyper-Kahler quotients of linear spaces. Let G be a compact Lie group with the 
Lie algebra g. Let M be its quaternionic representation. Let /, J, K denote multiplication 
by i, j, k, considered as linear operators on M. A quaternionic representation of G is 
a representation such that the G-action commutes with /, J, K. We suppose M has a 
G-invariant inner product ( , ) which is hermitian with respect to all I,J,K. Therefore 
M is a hyper-Kahler manifold with a G-action preserving the hyper-Kahler structure. We 
have the hyper-Kahler moment map p: M —» g* © M 3 , vanishing at the origin: 

(f, V{4>)) = ^((^0, 0), <i>), </>)), 

where cp G M, ( 6 g, and ( , ) is the pairing between g and its dual g*. A hyper-Kahler 
moment map is, by definition, (a) G-equivariant, and (b) satisfying 

where 0 is a tangent vector, £* is the vector field generated by £, and coi, coj, cok are Kahler 
forms associated with three complex structures /, J, K and the inner product. It is direct 
to see that two properties are satisfied in the above formula. 

In the following, we only use the underlying complex symplectic structure. Let us 
give another formulation. Let Gc be the complexihcation of G. Let M be its complex 
representation, which has a complex symplectic form ooc preserved by Gc- We have the 
complex moment map /x c : M —» gc 

c(0)) = 

where t; G gc- This is the complex part of the hyper-Kahler moment map under the 
identification M 3 = M © C. 

We consider the hyper-Kahler quotient 

M//G d = f ' ^\0)/G = ^\0)//G c , 

where //Gc is the affine algebro-geometric invariant theory quotient, and the second isomor¬ 
phism follows from a result of Kempf-Ness (see e.g., [Nak99, Th. 3.12]). Let ^i^ 1 (0) reg be the 
(possibly empty) open subset of /^ _1 (0), consisting of points with trivial stabilizers. Then 
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G acts freely on /x _1 (0) reg , and the quotient /x _1 (0) reg /G is a smooth hyper-Kahler mani¬ 
fold. The Higgs branch of the 3d N = 4 SUSY gauge theory Hyp(M)7^-G is /x _1 (0) reg /G or 
its closure in ^sA///G depending on the situation. We only consider Hyp(M) ff-G hereafter, 
but we actually mean fi~ 1 (0y eg /G when we talk the hyper-Kahler structure on it. 

The hyper-Kahler quotient IN A///G has a natural SU(2) = Sp(l)-action induced from the 
H-module structure of M. It commutes with the G-action, and hence descends to the 
quotient. It rotates the hyper-Kahler structure. In the complex symplectic notation, its 
restriction to U(l)-action is induced by the scalar multiplication on M. 

2(ii). Cotangent type. There is a class of quaternionic representations, which we call 
cotangent type. Let N be a complex representation of G. We put a G-invariant hermitian 
inner product on N. Then M = N © N* is a quaternionic representation of G. We define 
J by J(x,y) = (— y\x') for x £ N, y £ N*, where ad G N* is defined by (x\n) = (n,x) 
for n £ N, and y 1 is defined so that (ad)' = x for all x £ N. Then J is skew-linear as the 
hermitian inner product is skew-linear in the second variable. We have J 2 = — id from the 
definition. 

In the (complex) symplectic formulation, we start with a complex representation N of 
G(n, and take M = N © N* with a natural symplectic structure. Then M is naturally a 
representation of Gc, and has a symplectic form preserved by Gc- 

2(iii). Complete intersection. Although ^d(0)/G<c makes sense as an affine scheme 
without any further condition, we do not expect they behave well in general. We propose 
to assume 

• Mc 1 (0) a com pl e te intersection in M. 

More precisely it means as follows. We construct a Koszul complex from /x c : 

0 —> A dim0C 0c Om —>■■■—> A"0c ® Om ~> 0c ® Om o) 0. 

Our assumption says that this is exact. Under this assumption, the coordinate ring of 
/pA (0) is given by 

dim gc 

C[a©0)]= £(-1)7\‘bc®C[M]. 

i =0 

This is an equality of virtual C* x Gc-modules. Taking the Gc-invariant part, we get the 
Hilbert series of C[/x^ 1 (0)/Gc]- This is the definition used in the A'-theoretic Nekrasov 
partition function for instantons of classical groups. See [NS04]. 

We expect that this complete intersection assumption is related to the ‘good’ or ‘ugly’ 
condition, appearing in the monopole formula. But it is a superficial observation as we 
are discussing the Higgs branch now, while the monopole formula is about the Coulomb 
branch. 

2(iv). Quiver gauge theory. Let Q = (Qo,Qi) be a quiver, where Q 0 is the set of 
vertices and Q i is the set of arrows. We can construct a quaternionic representation of 
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G = Gy = UieQo U(Vi) associated with Qo-graded representations V = 0 V), W = 0 Wy 
It is a cotangent type, and given in the complex symplectic description by 

N = 0 Hom(K W , V m ) ® 0 Hom(Wi, V), 

M= ® Hom(K(/ l ),bi(fe)) © Hom(V r i W , V o{h) ) 

h(zQi 

© 0 Hom(Wi, Vi) © Hom(Vi, Wi). 

isQ o 

Here o(h) and i (h) are the outgoing and incoming vertices of the oriented edge h £ Qi 
respectively. 

If Q is the Jordan quiver, the hyper-Kahler quotient /^~ 1 (0 )/G is the ADHM description 
of the framed moduli space of SU(IH)-instantons on M 4 , or more precisely its Uhlcnbeck 
partial compactihcation (see e.g., [Nak99, Ch.3] and the reference therein). More gener¬ 
ally, hyper-Kahler quotient M ///G — /l^ 1 (0)/G ? c is the quiver variety (with complex and 
stability parameters 0), introduced in [Nak94], 

For a quiver gauge theory, Z(g*) = {(/ £ g* | Ad*/®) = Cm f° r any 9 G G} is nontrivial, 
and is isomorphic to 0, :e Q o Ma/— ltry, where try*: u(V)) —> /HI is the trace. Then we 
can form a perturbed hyper-Kahler quotient n~ 1 (()/G for ( £ M 3 ©Z( 0 *). If we decompose 
( as (Cm, Cc) according to M 3 = M © C, we have an algebro-geometric description 

M — Mc/Cc )//<; r Gc, 

where //^ is the GIT quotient with respect to the (/-stability. (See [Nak94, §3].) If £ is 
generic, ^ 4 ( O/G is a smooth hyper-Kahler manifold whose metric is complete. Moreover, 
we have a projective morphism 

Mc 1 (Cc )//(J3c = m 1 (0, Cc)/G. 

In an algebro-geometric approach to hyper-Kahler quotients, it is more natural to replace 
(/ by a corresponding element for the group G, i.e., a character y: G —> U(l), where they 
are related by (/ = d\- The character x defines a Gc-equivariant structure on the trivial 
line bundle over Mc^Cc)- We introduce the stability condition and form a GIT quotient 
with a natural projective morphism to Me* (Cc) HG: 

( 2 - 1 ) 7 r: 1 (Cc)/ x .Gc -» Mc 1 (Cc)/Gc 

It is equipped with a relatively ample line bundle C x in a natural way. See [Nak99, §3] for 
detail. 

In Figure 1 we follow physicists convention. The underlying graph of the quiver is 
circled vertices and edges connecting them. (An orientation of the quiver is not relevant, 
and usually omitted.) Dimensions of V) are put in circled vertices, while dimensions of W % 
are put in the boxed vertices, connected to the corresponding circled vertices. It is more 
or less the same as the original convention in [ N90]. 
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When W — 0, the scalar U(l) acts trivially, so we replace G by n«eQo U(Fi)/U(l). This 
is the case for M used by Kronheimer [Kro89], mentioned in Introduction. 

For a quiver gauge theory, good and ugly conditions were analyzed in [ W09, §2.4, §5.4], 
It is conjectured, for example, that a quiver gauge theory of finite type is good or ugly if 
and only if 

( 2 . 2 ) dim Wj — ^^( 2 Sij — a^) dim V) > —1 

j 

for any i G Q o - 1 Here a n] is the number of edges (regardless of orientation) between i and 
j if i ^4 j, and its twice if i = j. 

For quiver varieties n^r(Q) //Gc with W = 0, Crawley-Boevey [ 1501, Th.1.1] gave a 
combinatorial condition for the complete intersection assumption above. It can be modified 
to cover the W ^ 0 case using the trick [CB01, the end of Introduction], To state the 
result, let us prepare notation. Let C = (2 6^ — a,ij ) be the Cartan matrix. We denote the 
dimension vectors (dim Vi)j e <g 0 , (dimWj)j e Q 0 by v, w respectively. A root is an element 
of obtained from the coordinate vector at a loopfree vertex or ± an element of the 
fundamental region by applying a sequence of reflections at loopfree vertices. If there are no 
loops, this notion coincides with the usual notion of roots of the corresponding Kac-Moody 
Lie algebra by [Kac90, Th. 5.4], 

Then / n^ 1 (0) is a complete intersection if and only if the following is true: 

• *v(2w — Cv) > *v°(2w — Cv°) + J0 fc (2 — for any decomposition v = 

v° + such that w — v° is a weight of an irreducible highest weight module 

V(w) of the highest weight w, and /3 ^ is a positive root. 

(cf. [Nak09, Th.2.15(2)] for a closely related condition.) The dominance condition (2.2) is 
a necessary condition, from the decomposition v = v° + cq, but not a sufficient if there is 
with *£( fc )C/3W < 2 , i.e., an imaginary root. Anyway these two conditions are closely 
related. This is the reason why we expect the complete intersection assumption and the 
‘good or ugly’ condition are the same. 

2(v). Type A quiver, nilpotent orbits and affine Grassmannian. As a special class 
of quiver gauge theories, type A (or linear) quiver gauge theories are important. The 
hyper-Kahler quotient, in other words, Higgs branch was identified with O M fl S\, where 
O^ is a nilpotent orbit and S\ is Slodowy slice to another orbit 0\ of type A (see [Nak94, 
§ 8 ]). Here we assume /x _ 1 (0) reg 7 ^ 0, and two partitions A, fi are defined from dimensions 
of V, W by an explicit formula . 2 Conversely any O M fl S\ for type A is described as a 
hyper-Kahler quotient. 

Let us recall how the identification is constructed. The starting point is Kronhcimer’s 
realization [ \ro90] of O^ fl S\ as moduli spaces of SU(2)-equivariant instantons on M 4 . 
This construction works for any compact Lie groups. For type A, we apply the ADHM 
transform to these instantons. Suppose that an instanton corresponds to (Hi,i? 2 ,a, b) G 


4 Tlie conjecture is stated only for goodness, and the right hand side is replaced by 0 in [GW09]. 

2 There is typo in [Nak94, §8]. ^ must be replaced by its transpose. 
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M = Hom(V, F)® 2 © Horn (W,V) © Hom(F, IF), the data for the Jordan quiver. If the 
original instanton is SU(2)-equivariant, V, W are representations of SU(2), and (Bi,B 2 ), 
a, b are SU(2)-linear. Here we mean the pair (Bi,B 2 ) is SU(2)-equivariant, when it is 
considered as a homomorphism in Hom(V, V® p 2 ), where p 2 is the vector representation of 
SU(2). Let pi be the ^-dimensional irreducible representation of SU(2). We decompose V, 
W as 0 Vi®pi, 0 W t ®pi. Then a, b are maps between Vi and W % . By the Clebsch-Gordan 
rule Pi®p 2 = Pi-i © Pi+i, (Bi, B 2 ) decomposes into maps between V t and F)-i © V i+ i. The 
dimensions Vi, Wj are determined by A, p, as mentioned above. In a nutshell, the McKay 
quiver for SU(2) is the double of type A^ Dynkin graph (figure 2). Hence we get a quiver 
variety of type A^. 




Figure 2. McKay quiver for SU(2) 


A quiver variety of type A can be obtained also from a framed moduli space of S 1 - 
equivariant instantons on R 4 , where S 1 acts on R 4 = C 2 by t ■ (x,y) = (tx,t~ lr y). The 
reason is the same as above: (a) irreducible representations pi of S 1 are parametrized 
by integers i 6 Z, i.e., weights, and (b) pi ® C 2 = p*_i © Pi+\, where C 2 is the base 
manifold, identified with p\ © p_i as an A'-module. Strictly speaking, McKay quiver for 
S 1 is slightly different from one for SU(2), and inhnite in both direction (figure 3). But 
the quiver varieties remain the same as V is finite-dimensional. 



Figure 3. McKay quiver for S 1 


By [BF10, §5] the framed moduli space of S''-equivariant G-instantons on R 4 is also 
identified with the intersection Wg A of a G[[^]]-orbit Gr(f in the affine Grassmannian 

Grc = G((^))/G[[z]] with a transversal slice to another G[[z]]-orbit Gr^. Here we regard 
A, p, as homomorphisms S 1 —> G, i.e., coweights of G. 3 This result is true for any G. 
Note that the identification of partitions with coweights in [ F10] is different from one in 
[Nak9 I, §10], which will be used below. 


'’Here A (resp. p) corresponds to a homomorphism at oo (resp. 0) of C 2 . Since we follow the convention 
in [ ak94], this is opposite to [BF10]. In particular, Wq x is empty unless A < p. 
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Thus we have identifications 

q p| § _ y a framed moduli space of SU(2)-equivariant 

M G'-instantons 

t 

a quiver variety of type A 

t 

a framed moduli space of S' 1 -equivariant ^_ y yyM 

G-instantons G,A " 

The identifications O tl fl S\ and Wq x with quiver varieties of type A were first found in 
[MV03]. Note however that horizontal arrows remain true for arbitrary G, while vertical 
ones are true only for type A. In the top row A, p are nilpotent orbits, while they are 
coweights in the bottom row. Therefore we cannot hope a vertical relation for general 
G. Therefore one should understand O f , fl S\ = Wq A as a composition of the natural 
horizontal identifications and accidental vertical ones. If we apply the ADHM transform 
to SU(2) and S' 1 -equivariant instantons for classical groups respectively, we will obtain 
different modifications of quiver varieties. It will be discussed in §§§A(i) ,A(ii) , A(iii) . For a 
finite subgroup T C SU(2), we can also consider T-equivariant instantons in the same way. 
See §A(iv). 

The mirror of this theory is given by 0\t fl where A f , pf are transpose partitions. (It 
is not clear at this moment, what is the mirror if /x _1 (0) reg = 0, i.e., p is not necessarily 
dominant.) This mirror symmetry can be naturally extended to the case of quiver gauge 
theories of affine type A. It nicely fits with the level-rank duality of affine Lie algebras of 
type A via the author’s work [Nak94], This was observed by de Boer et al [dBHO + 97, §3] 
based on brane configurations in string theories introduced by Hanany-Witten [ IW97]. 
Further examples of the mirror symmetry will be given in §3. 

2(vi). Abelian theory. Let us take a collection of nonzero integral vectors u\, ..., Ud in 
Z n such that they span Z”. We have an exact sequence of Z-modules 

(2.3) 0 -A Z d ~ n AZ rf Az n A0, 

where f3\ 7L d —>■ Z n is given by sending the coordinate vector e; to Ui, and the kernel 
of Z d —> Z n is identified with 7L d ~ n by taking a base. We have the corresponding exact 
sequence of tori: 

(2.4) 1 -A G = U(l) d - n 4 T d = U(l) d Ag f = U(l) n -A 1, 

where maps in (2.3) are induced homomorphisms between coweight lattices (or equivalently 
fundamental groups). 

Let M = and let T d act H-linearly on M by multiplication. We consider Hyp(M)^G. 
It is called the abelian theory. It is of cotangent type with N = C d . 

The hyper-Kahler quotient MjfG is called a toric hyper-Kahler manifold and was in¬ 
troduced by Biclawski and Dancer [ !D00]. Note that we have an action of Gp on M ///G. 
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This group is called a flavor symmetry group, and its importance will be explain in §5 
below. 

The space Z(g*) is nontrivial as in quiver gauge theories, and /^ _1 (CimH[)/G is a hyper- 
Kahler orbifold for generic CimH- 

The abelian theory is a good example to understand the 3d mirror symmetry. We dualize 
the exact sequence (2.3) to get 

1 _► G w f ^ ( T d Y 4g v a1, 

where « v denotes the dual torus, defined by 7 Ti(«) a . (G v is also U(l) d-n , but we would 
like to make our framework intrinsic.) Then we can consider another toric hyper-Kahler 
manifold M/G)^ with a G v -action. ft was proposed in [iBHO + 97, §4] that Hyp(M) -fff-G 
and Hyp(M) -ffj- G y F form a mirror dual theories. In particular, the Coulomb branch of 
Hyp(M) fff-G is yi///G W F . 


3. More on 3d mirror symmetry 

It is important to have many examples of 3-dimensional mirror symmetric pairs, as they 
determine the Coulomb branches as the Higgs branches of mirror theories. We give more 
examples in this section. 

3(i). Mirror could be a non-lagrangian theory. We first remark that the mirror of 
the gauge theory Hyp(M) -jff-G may not be of a form Hyp(M') -fff-G' for some M' and G' in 
general. For example, if we replace the diagram in Figure 1 by the affine Dynkin diagram 
of type E^l 8 as in Figure 4, it is expected that M.c is H ie framed moduli space of E 6 j yS 
h-instantons on M 4 . This example was found in [ ] for k — 1, and in [ ] for 

general k. It is widely accepted a common belief that there are no ADHM like description 



Figure 4. A 4c ■ E s fc-instantons on M 4 . 

of instantons for exceptional groups. It means that the moduli spaces cannot be given 
by a hyper-Kahler quotient M 'jljG' (with finite dimensional M', G'), hence the mirror 
theory B is not of a form Hyp(M') -ffj- G'. In fact, the mirror theory B is known as a 3d 
Sicilian theory [BTX10], which does not have a conventional lagrangian description. See 
§3(iii) below. Nevertheless we can compute Hilbert series of instanton moduli spaces of 
exceptional types by the monopole formula. 4 This is even more exciting, as there is only a 
few way to compute them, say a conjectural blowup equation [ Y05a, NY05b], 

4 One need to modify the monopole formula to deal with non simply-laced groups. See [CFHM14], 
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3(ii). Instantons on R 4 /r. Let us consider a quiver gauge theory of affine type. An 
affine quiver of type ADE arises as the McKay quiver of a finite subgroup T of SU(2). 
Hence the Higgs branch, the hyper-Kahler quotient of M by G. parametrizes T-equivariant 
U(t')-instantons on R 4 as in §2(v). In fact, this is a starting point of the work [ vN90], 
which eventually leads to the study of quiver varieties [Nak94], 

As we have mentioned already in §2(v), the mirror of a quiver gauge theory of affine 
type A is another quiver gauge theory again of affine type A. The precise recipe was given 
in [dBHO+97, §3.3]. 

From this example, together with Braverman-Finkelberg’s proposed double affine Grass- 
mannian [BF10], we will give an initial step towards the determination of the mirror of 
the quiver gauge theory of an arbitrary affine type as follows. It was mentioned in a vague 
form in [ PW14, Rem. 10.13]. 

A framed moduli space of T-equivariant G -instantons on R 4 has discrete data, a usual 
instanton number, as well as po,Poo'- T —> G homomorphisms from T to G given by T- 
actions on fibers at 0 and oo. A quiver variety, that is the Higgs branch of a quiver gauge 
theory of affine type, corresponds to the case G — U(T). We regard p 0 , p^ as ^-dimensional 
representations of T. They are given by dimension vectors (dim Vi), G Q 0 , (dim W{)i e Q 0 by 

Poo = © p® dimWi , p 0 = (J) pf Ui with m = dim W t - J^(2 6# - a tJ ) dim Vj, 

j 

where {p*} is the set of isomorphism classes of irreducible representations of T, identified 
with Q 0 via McKay correspondence. If some tq is negative, there is no genuine instanton. 
In other words, the Higgs branch JA H = M ///G contains no free orbits. We do not know 
what happens in the Coulomb branch without this assumption. Conversely tq determines 
dim V modulo the kernel of the Cartan matrix C, i.e., ZS for the (primitive) imaginary 
root 5. This ambiguity is fixed by specifying the instanton number as JA 5j dim Vj where 
5j is the j th -entry of 5. 

Let g be the complex simple Lie algebra of type ADE corresponding to T. Let 0 a ff be 
the associated untwisted affine Lie algebra, containing the degree operator d. In [Nak94], 
an affine Lie algebra representation was constructed by quiver varieties. Dimension vectors 
give affine weights of g aff by 

A = y^(dim Wi)Ai, p = ^(dirn Wj)A* - (dim 1 7i)a u 

i i 

where A* (resp. cq) is the i th fundamental weight (resp. simple root). The weight A is 
always dominant. We also have A > p by definition. If we assume rq > 0 for all i as above, 
the second weight p is also dominant. 

On the other hand, Braverman and Finkelberg [BF10] associate a pair (A > p) of affine 
weights with po, Poo and instantons numbers as follows. They take T = Z/IZ, where d is 
the level of A, also of p as A > p. They take G a simply-connected group, possibly of type 
BCFG. Then [ F10, Lemma 3.3] says a conjugacy class of a homomorphism Z/IZ —>• G 
corresponds to a dominant coweight A of Gc with (A, 6) < £, where 6 is the highest root of 
g. It can be regarded as a level i weight of G^, the Langlands dual of the affine Kac-Moody 
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group Gc- Here Gf- does not contain the degree operator. We assign dominant weights 
A, p of level i to po, p 0a respectively in this way. Then we extend them to A, p dominant 
weights of the full affine Kac-Moody group Gf n so that 


instanton number = £(\ 


P, d ) + 


(A, A) 
2 


(/b h) 

2 


See [1 5F10, (4.3)]. This rule only determines (A — p,d), but it is well-known that represen¬ 
tation theoretic information depend only on the difference A — p. 

Thus a pair of affine weights (A > p) correspond to instanton moduli spaces in two ways, 
when G is of type ADE , first in [ \ak94] , second in [ LO] , as we have just explained. Take 
the quiver gauge theory whose Higgs branch is the quiver variety associated with (A > p) 
in the first way. Then its Coulomb branch is expected to be the Z/£Z-equi variant instanton 
moduli space associated with (A > p) in the second way. 

However this is not precise yet by the following reason. Since affine weights of the Lie 
algebra g a ff may not give weights of G)( ff if G is simply-connected, we need to replace 
G by its adjoint quotient. Then [ F10, Lemma 3.3] says a homomorphism Z/tZ —>• G 
corresponds to an element in the coset A/FF a ff^, where A is the coweight lattice of G , and 
W a ff £ is the semi-direct product IF ix tt of the Weyl group W and A. Here £A acts on A 
naturally. If G is of adjoint type , A is the weight lattice of G v , i.e., the weight lattice of 53 . 
But W a ff t £ is an extended affine Weyl group, i.e., the semi-direct product of the ordinary 
affine Weyl group and a group T consisting of affine Dynkin diagram automorphisms. 
Then a point in the coset A/W a does not give an affine weight of Gf-. It only gives a 
T-orbit. 

When G is of type A r _i, this inaccuracy can be fixed: we replace G by U(r), and (A > p) 
by a pair of generalized Young diagrams, in other words, dominant weights of GL(r, C). 
See [Nak09, App. A] for a detailed review. If we view both Higgs and Coulomb branches as 
quiver varieties of affine type A, the rule of the transform of dimension vectors is given by 
transpose of generalized Young diagrams, as reviewed in [Nak09, App. A], It is the same 
as one in [ IO + 97] up to a diagram automorphism. 

If we take a gauge theory of finite type instead of affine type, A, p are dominant weights 
of the finite dimensional Lie algebra g in [^Jakhd]. Then instead of [ F10], one can use 
just the ordinary geometric Satake correspondence, i.e., the affine Grassmannian for G of 
adjoint type. In terms of instantons, we use S' 1 -equivariant G-instantons on M 4 . Then A, 
p are regarded as dominant coweights of G, and correspond to S' 1 -actions on fibers at 0 
and 00 . The inaccuracy disappears also in this case. This conjectural proposal was given 
in [ PW14, Rem. 10.7], though their symplectic duality does not make sense in general 

outside type A, as we mentioned in Introduction. 


3 (iii). Sicilian theory. There is another class of 4-dimensional quantum held theories with 
Af = 2 supersymmetry. They are called theories of class S. A theory Sr(C, aq, pi ,..., x n , p n ) 
is specified with an ADE Dynkin diagram T, a punctured Riemann surface (G, aq,..., x n ) 
together with a homomorphism pp. su(2) —>- g r for each puncture aq, where gr is the Lie 
algebra of a compact Lie group of type T. It is constructed as a dimensional reduction of a 
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6-dimensional theory associated with h, compactihed on a Riemann surface C with defects 
at punctures specihed by p % . It is believed that Sr(C, aq, p\,... ,x n ,p n ) does not have a 
lagrangian description in general, hence is not a gauge theory studied in this paper. See 
[Tac] for a review aimed for mathematicians. 

Physicists consider its Coulomb and Higgs branches. The Coulomb branch is expected 
to be the moduli space of solutions of Hitchin’s self-duality equation on C with boundary 
condition at aq given by pi. On the other hand, it is asked in [ ] what is the un¬ 

derlying complex symplectic manifold of its Higgs branch A4 H (Sr{C, x±, pi,..., x n , p n )). 
The underlying complex symplectic manifold is independent of the complex structure of 
(C, xi,... , x n ). It is supposed to satisfy various properties expected by physical considera¬ 
tions, most importantly it gives a 2d TQFT whose values are complex symplectic manifolds. 

We can further compactify Sr(C, aq, p \,..., x n , p n ) on S^ and take limit R —> 0 to get 
a 3-dimensional quantum held theory with AT = 4 supersymmetry. This is a 3 d Sicilian 
theory mentioned above. It is expected that the Higgs branch is unchanged under the 
compactihcation by S^. 

When T is not exceptional, its mirror is supposed to be a certain gauge theory Hyp(M)^ 
G [ 5TX10]. Therefore the Higgs branch M.h{St{C,X\, /q,... ,x n , p n )) is the Coulomb 
branch of a gauge theory _Mc(Hyp(M) iff- G ), which we are studying in this paper. 

Let us specify M and G. First suppose T is of type Ag. If C = S 2 , the mirror is the 
quiver gauge theory associated with the star shaped quiver with n legs. Entries of the 
dimension vector are l at the central vertex, and given by pi on the i th leg specihed by 
the rule as for the quiver construction of the nilpotent orbit /q((j q) ( see §2( v ))- If C has 
genus g , we add g loops at the central vertex. (Figure 5) Note also that quivers considered 
in §3(i) are of this type. For type De, we modify this quiver as in §A(ii). 



Figure 5. Mirror of a 3d Sicilian theory of type An 

The Coulomb branch p\,... ,x n ,p n )) of a 3d Sicilian theory is the Higgs 

branch of the gauge theory, which is the hyper-Kahler quotient M ///G. It is an additive ver¬ 
sion of the moduli space of homorphisms from the fundamental group 7r 1 (C \ {^i ,... , x n }) 
of the punctured Riemann surface to GL£ +1 (C) or SO(2£, C) with prescribed conjugacy 
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classes around punctures. (See [CB03].) There is an isomorphism between an open subset 
of M///G and the actual moduli space [Yam08], hence it is compatible with the expectation 
that M.c(Sr(C,xi, pi,... ,x n , p n )) is the Hitchin moduli space. When we make R —> 0, 
the Hitchin moduli space is replace by its additive version. 

The monopole formula for this type of quivers is studied in [ MZ14b], 

4. Monopole formula 

We discuss the monopole formula in detail in this section. 

4(i). Definition. Let G be a compact Lie group. We assume G is connected hereafter 
for simplicity. 5 We choose and fix a maximal torus T and a set A + of positive roots. Let 
Y = Y(T ) be the coweight lattice of G. Let W denote the Weyl group. 

Suppose that a quaternionic representation M (also called a pseudoreal representation) of 
G is given. We choose an H-base {b} of M compatible with the weight space decomposition. 

We define two functions, 6 one depending on G and M, another depending only on G, of 
a coweight A G Y by 

A(A) d = - £ l<a,A}| + )D<wt(UA)|, 

(4.1) oGA+ b 

A) ^ n 1^57- 

where ( , ) is the pairing between weights and coweights, and the product in the second 
formula runs over exponents of the stabilizer Stabc(A) of A. Since we take the absolute 
value in the second term, A (A) is independent of the choice of b: it remains the same for 
jb. It is well-known that Pcj(t] A) is equal to the Poincare polynomial of the equivariant 
cohomology ^stab G (A) (pt) °f a point. Both A(A) and Pc{t: A) are invariant under the Weyl 
group W action on Y. 

We assume 2A(A) > 1 for any A ^ 0, a ‘good’ or ‘ugly’ theory in the sense of [ 1W09], 
hence no negative powers of t appear. (It is good if 2A(A) > 1 and ugly if 2A(A) > 1 and 
not good. But we do not see any differences of two conditions in this paper.) Since A(A) 
is piecewise linear, there is only finitely many A for a given 2A(A). Note also that Pc(t'- A) 
can be expanded as a formal power series in t. Therefore 

H GM (t) d = t2AW PG(t-, A). 

\£Y/W 

makes sense as a formal power series in t. 

This elementary, but combinatorially complicated expression is the monopole formula 
for the Hilbert series of the Coulomb branch M. c = A4c , (Hyp(M) -ffj-G). 

Recall that physicists claim that the Coulomb branch Ai c is a hyper-Kahler manifold 
with an SU(2)-action rotating hyper-Kahler structures /, J, K. One choose a complex 
structure /, and take U(l) C SU(2), fixing /. Then the Hilbert series is the character 


5 The monopole formula for a disconnected group O (N) appears in [CHMZ15]. 

following [CFHM14], we change t by t 2 from [ HZ14], 
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of the coordinate ring C [M.c\ of M-c with respect to the U(l)-action, endowed with an 
affine scheme structure compatible with the complex structure I. Thus the main claim in 

[CHZ14] is 

Hg, m(£) = chu(i) C[A4c]- 

Remark 4.2. The good or ugly condition 2 A(A) > 1 for any A 7 ^ 0 means weights of C[A4c] 
are nonnegative and the 0-weight space consists only on constant functions. If this is not 
satisfied, it is not clear whether (1.2) makes sense or not. However C[A 4c\ itself might be 
well-defined. It is the case for Hyp(0)//SU(2) for example. The only trouble is that weight 
spaces might be infinite dimensional, and hence the character chum C[Alc] is not defined. 

4(ii). Examples. Let us calculate Hq, m(^) for the simplest example. Let G — U(l) and 
M = HI = C © C*, the vector representation plus its dual. We identify the coweight lattice 
E(U(1)) with Z, and denote a coweight by m instead of A. 

There is no first term in A (m) as A + = 0. Thus 2A(m) = \m\. This is an ugly theory. 
The stabilizer of m is always U(l), thus Pc(t',m) = 1/(1 — t 2 ). Therefore 

tH = (l _f)2' 

In this case, Seiberg-Witten [ !W97] claim that the Coulomb branch A4c is the Taub-NUT 
space. It is a 4-dimensional hyper-Kahler manifold with Sp(l)-action, whose underlying 
complex manifold is C 2 . The subgroup commuting with the complex structure / is U(l) 
with the multiplication action on C 2 . Therefore the Hilbert series is 1/(1 — t) 2 as expected. 
If we replace M by the direct sum of its IV-copies, i.e., M = M N , we get 

1-1 -t N 1 _ t 2N 

( 4 - 3 ) 

It is claimed that A Ac is the multi-Taub-NUT space, which is C 2 /(Z/1VZ) as a complex 
variety. It is the surface xy = z N in C 3 , hence we recover the above formula if we set 
deg x = deg y = N, deg z — 2. 

In this case, the meaning of individual terms t 2 A Wp G (t; A) = t iV l m l/( 1 — t 2 ) is also 
apparent from the exact sequence 

0 -A C[M c \ 4 C[M c \ -> C[{z = 0}] -a 0. 

We have (1 — t 2 ) chup) C[A ic\ = chu(i) C[{x; = 0}]. This explains Pg{R A) = 1/(1 — t 2 ). 
Since {z = 0} = {xy = 0}, we have C[{£ = 0}] = C[x,y\/(xy). Then t N l m l corresponds to 
x m for m > 0 and y m for m < 0 (and 1 for m = 0 ). 

4(iii). Expected properties of A ic- Let us give several expected properties of the 
Coulomb branch A ic- First of all, 

(a) M.c contains a hyper-Kahler manifold (or orbifold, more generally) with an SU(2)- 
action rotating /, J, K , as an open dense subset. 

Once the Hilbert series is given, the dimension is given by the degree of the corresponding 
Hilbert polynomial. We expect 
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(b) dim-Ht Me = dim R T. 

It is a classical result that the fundamental group 7 Ti (G) of G is isomorphic to the 
quotient of the coweight lattice Y by the coroot lattice. We can refine the Hilbert series 
by remembering the class of A in 7 Ti (G), i.e., the coordinate ring C[A4c] has an additional 
7 Ti (G')-grading. Therefore we expect 

(c) The Pontryagin dual 7r 1 (G) A = Hom( 7 r 1 (G), U(l)) acts on M. c , preserving the 
hyper-Kahler structure. 

In [ HZ14, CHMZ14a], an additional variable z is introduced, and (1.2) is refined to 
(4.4) H GM (t,z)= ^ J(A) t 2A(A) P G (t; A), 

\£Y/W 

where J is the projection from Y to vr 1 (G). In above examples, G is a product of unitary 
groups, hence, 7 Ti(G) = 7L r . Thus we expect the torus U(l) r acts on M. c . In the above 
formula, z is a (multi) variable for characters of the torus. 

Let us check these expected properties for the abelian case, (a) is clear as Wfc is supposed 
to be the hyper-Kahler quotient M ///G'p. We have 

(b) diniH M ///G y F = d — n = clinic G. 

(c) vri(G) A = ( Z d ~ n ) A = G w acts on M///G V F . 

Therefore these two proposed properties are satisfied. 

In many examples, this group action can be enlarged to a nonabelien group action. Let 
us give a particular example. 

Take a quiver gauge theory Hyp(M) Hf- G as in §2(iv). Since G is a product of unitary 
groups, 7 Ti(G) a is isomorphic to the product of U(l) for each vertex i G Q 0 . Therefore 
U ieQo U(l) acts on M. c from (c) above. It is expected that a larger group containing 
U ieQo U(l) acts on A4c as follows. 

We consider the Weyl group W(Q), naturally appeared in the context of quiver varieties 
[Nak94, §9]: Fix W, or more precisely dim W = (dim Hq) G Z^°, but we allow V to 
change. We regard (dim Wi — ~ a ij ) dimV^) G T9° as a weight of the Kac-Moody 

Lie algebra corresponding to Q. Then we change dim V given by the usual Weyl group 
action on weights. Concretely, for each vertex i without edge loops, we consider s,, which 
change V by a new V' by the following rule: A) V- is the same as V 3 if j ^ i. B) 
dim V- = dim W, + ^ . a i3 dim Vj — dim V). These .s t generates the Weyl group. 

Now we fix V again, and introduce the subquiver S = (So, S i) of Q consisting of vertices 
i such that reflections s* preserve dim V and edges between them. We suppose S is of 
finite type, i.e., the underlying graph is a disjoint union of ADE graphs. Let G' s be 
the simply-connected 7 compact Lie group corresponding to S. Note that G' s contains 
U(l)’s corresponding to vertices in S, as a maximal torus. We then take other U(l)’s 
corresponding to vertices not in S, and define Gs as the product of G' s and those U(l)’s. 
Now vri(G) A = EL e0o U(l) is a maximal torus T$ of Gs- 

'The author does not know whether it naturally descends to a quotient or not. 
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We also consider the group T of the diagram automorphism of preserving both dim V, 
dim W. ft acts on Gs by outer automorphisms. Then we expect 

(d) T x Gs acts on Xic compatibly with the 7 Ti (G) A -action, preserving the hyper-Kahler 
structure. 

In the above example in Figure 1, the left one contains a type Ajy _i subgraph in the 
bottom. Therefore Gs = SU(iV) x U(l), where the extra U(l) comes from the upper 
circled vertex. We also have the overall U(l) in SU(2), and T = {±1} from diagram 
automorphisms. Thus their product should act on Xi^. This should be the same as the 
natural action on Xip, the framed moduli space of SU(iV)-instantons on R 4 , where SU(iV) 
acts by the change of framing, U(l) x U(l) acts on the base R 4 = C 2 with ( t , z ) • (or, X 2 ) = 
(tzxi, tz~ l X 2 ), and the T-action is given by taking dual instantons. In the example in 
Figure 4, we have E% x U(l) x U(l)-action from this construction. 

Remark 4.5. Let us write ni(G) = Y/Y cr , where Y (resp. Y cr ) is the coweight (resp. coroot) 
lattice of G. Therefore 7 r 1 (G) A is the kernel of the homomorphism Y A —>■ Yfl between 
Pontryagin duals of Y , Y CI . We have Y A = (A" <g) R)/X, where X = Hom(Y, Z) is the 
weight lattice of G. Therefore the coweight lattice Hom(U(l), 714 (G) A ) is the same as the 
kernel of the homomorphism X —> Hom(Y cr , Z) given by the pairing with coroots Y cr . It is 
equal to the character group of G. I 11 summary, we have a natural isomorphism 

(4.6) Hom(U(l), vri(G) A ) =* Hom(G, U(l)) ^ Hom(G c , C*). 

Thus a coweight x G Hom(U(l), 7 Ti(G) a ) defines a character G U(l), and hence gives 
a stability condition and the corresponding GIT quotient of /^^(Cc) by Gc as in (2.1). 

Therefore an element of (4.6) plays two roles, one on A in, another on Xic- This 
observation was essentially given already in [ S96, dBH0097]. The former appears as 
a value of the hyper-Kahler moment map, or Fayet-Iliopoulos parameter in the physics 
terminology. On the other hand, when Gs acts on Xic, an element of the Lie algebra of 
Gs is called mass parameter. 


5. Flavor symmetry 

5(i). Line bundles over Coulomb branches. Let us discuss a flavor symmetry following 
|CHMZ14a]. It means that we suppose that M is a quaternionic representation of a larger 
compact Lie group G, which contains the original group G as a normal subgroup. The 
quotient Gp = G/G is called the flavor symmetry group. For a quiver gauge theory, we 
can take (at least) Gf = IlieQo U(W)/ U(l) (and G = G x Gf), where U(l) is the overall 
scalar, which acts trivially on M jj/G. 

Note that we do not use Gp to take a quotient. The gauge theory Hyp(M) jjj- G has 
a G^-symmetric QFT in the sense of [Tac]. (The reader needs to remember that M is 
a representation of G.) As a concrete mathematical consequence, for example, Gf acts 
as a symmetry group on the Higgs branch A4^(Hyp(M) jff- G), which is the hyper-Kahler 
quotient M ///G. 

Let us turn to study the role of Gp playing on the Coulomb branch _Mc , (Hyp(M) -ffj-G). 
As observed in [ IMZ14a], we can naturally put Gp in the monopole formula ( 1 . 2 ) as 
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follows. Let us consider the short exact sequence of groups 


1aGaGAg f a1. 


(This is the same as (2.4) in the abelian case.) Let us fix a coweight A p of Gf and 
consider the inverse image /3 _1 (Af). We consider A(A) and Pc(i; A) for A G /3 _1 (A p). 
Their definitions in (4.1) remain the same. For A(A), the first term is the sum over A + , 
positive roots of G, considered as positive roots of G. In the second term, we understand 
wt(6) as a weight of G, and paired with A. We do not change Pc(t] A), we consider the 
stabilizer of A in G, not in G. The sum is over /3 —1 (A f)/W , where W is the Weyl group of 
G. We get a function 


Hot «Mf) = £ « 2 A < A >P G ((i A) 


AS/S-qA F )/W 


in t together with Xp. 

It was found in [ \lZ14a] (and more recent one [CHMZ15]) that this generalization 

turned out to be very fruitful by two reasons: 

First, let {Hyp(Mj jff- Gj)}^!^,... be a collection of simpler gauge theories sharing the 
common flavor symmetry group Gp. (Thus M, ; is a representation of G i; and Gp = Gi/Gi.) 
We define a complicated gauge theory as Hyp(0 M 4 -jff- ]0 Gf), where ]0 G* is the fiber 
product of UGi and the diagonal Gp over The Hilbert series of the complicated 

theory is written by those Hg. M .(i; A p) (i = 1, 2,3 ...) of simpler theories as 



a f g y f /w f 


where Yp, Wp , A^ are the coweight lattice, Weyl group, the set of positive roots of Gf. 
This is clear from the form of the monopole formula. 

Second, if Hyp(M iff- G) is a quiver gauge theory of type A (or its Sp / O version), the 
Hilbert series are written by Hall-Littlewood polynomials. 

Combining two, one can write down the Hilbert series of Higgs branches of 3d Sicil¬ 
ian theories in terms of Hall-Littlewood polynomials, as an example of an application 
[CHMZ14b], 

Since quiver varieties of type A are nilpotent orbits (and their intersection with Slodowy 
slices) as we mentioned in §2(iv), the appearance of Hall-Littlewood polynomials is very 
suggestive. They appear as dimensions of spaces of sections of line bundles over flag 
varieties. See [Bro93], where the Euler characteristic version was found earlier in [Hes80]. 

For an abelian gauge theory, the dual torus of Gf appears in the quotient construction 
of the Coulomb branch as A4 C = M ///Gf (§2(vi)). Therefore a coweight A p of Gf, which 
is a weight of Gf, defines a line bundle over the resolution /Gf as 


H 1 (ClmlHl) X G v C, 


where G'p acts on C by A^. It naturally has a connection, which is integrable for any of I, 
J, K [GN92]. In particular, it is a holomorphic line bundle with respect to I. 
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Based on these observations, it is natural to add the followings to the list of expected 
properties: 

(e) We have a (partial) resolution of Me whose Picard group is isomorphic to the 

coweight lattice Yp of Gp. Moreover the character of the space of sections of a 
U(l)-equivariant holomorphic line bundle C\ p corresponding to a coweight \ F is 
given by the monopole formula M (f, A^). (Here we replace Me if necessary so 

that C\ F is relatively ample.) 

(f) The Weyl group Wp acts on the Picard group of the partial resolution above. 

As is usual for hyper-Kahler manifolds, a resolution and a deformation are related by 
the hyper-Kahler rotation. Therefore we expect 

(g) We have a deformation of Me parameterized by the Cartan subalgebra fyp of Gp. 
The Weyl group Wp acts on the homology group by the monodromy. 

Let us check the compatibility of the conjecture with the 3 d mirror symmetry. Recall 
that the group V x Gs acting on Me (see §4(iii)(d)). Since the definition of V x Gs depends 
on the choice of the theory A, let us denote it by T 4 x Gg. It is natural to expect that it 
is identified with Gp (the flavor symmetry group for the theory B) acting on Mfj. This 
is indeed the case for the example in Figure 1, where T' 4 x Gg are {±1} tx SU(iV) x U(l). 
The squared N gives us SU(iV). The factor U(l) comes from an internal symmetry of the 
graph. The edge loop at the circled k gives the factor End(C fc ) ® C 2 in M. Then U(l) 
is acting on C 2 preserving the hyper-Kahler structure. 8 Finally {±1} is identified with 
the symmetry defined by transpose of linear maps in M. (Since M is the cotangent type, 
transpose of an element in N is in N.) 

Since the mirror symmetry should be a duality, the role of Gp in Me should be the 
same as the role of T' 4 x Gg playing in Mjj. The diagram automorphism group T" 4 induces 
automorphisms on Mfj. This is clear. 

Let us turn to Gg. Recall that we have defined Gg in two steps. We first define the 
maximal torus of Gg as 7 t 1 (G) a , where the group G is the one we take the quotient. Then 

we consider the Weyl group invariance in the second step. So consider Tg d =' 7 Ti(G) a 
first. By (4.6) a coweight A g e Hom(U(l), Iff) defines a character of G, and hence gives a 
(partial) resolution and a line bundle on it for the Higgs branch M^ as in (2.1). This is 
exactly the property (e) for Gf and Mq. 

Moreover, if we take A g generic in Hom(U(l), Tg), we expect that the Picard group of 
Akc 1 (0) H\ s Gc is isomorphic to Hom(U(l), Tg) and we have an action of the Weyl group of 
Gg, as in (f). 

5(ii). Abelian case. In [ , §6], the proposal (e) was checked for the abelian theory. 

Let us give a different proof. 

We keep the notation in §2(vi). We choose and fix a coweight X F of Gp, considered 
also as a weight of Gf. We define the Coulomb branch Me by M j/fG'p and its partial 
resolution Me by /^ _1 (Af, 0 )/G' F = Rc^O)/^ (G F )c- Here A^ is considered as (LieG)()* in 


8 A larger group Sp(l) acts on C 2 , but the author does not know how to see it in the monopole formula. 
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the first description as a hyper-Kahler quotient, and the stability parameter in the second 
description as a GIT quotient. (The complex parameter is set 0.) We have a relative ample 
line bundle C\ F = ^ _1 (A F , 0) Xgv C. Our goal is to check that the character of the space 
of sections of C\ F is given by P GM (t, A F ). 

In order to have a clear picture, we consider the action of tti(G) a = G v on A4 G . We 
take a lift of the action to the line bundle C\ F . It means that we lift Ac to a weight \ F of 
(' T d ) w so that we have the induced G v = (T d y /G y F -action on /x _1 (Ac, 0) x G v C. Then the 
space of sections is a representation of U(l) x G v . The monopole formula is refined as 

(5.1) H aM (t, X F )= Y, T« 2A « a >P g ((; A), 

Aez d ~" 

as in (4.4). Here we use the identification /3 _1 (Ac) = X F + Ima = Z d ~ n , as we choose the 
lift Ac- We understand z as a multi-variable, and means 1 • • • 

Since G is torus, the stabilizer of A is always G itself. Therefore P G (t; A) = 1/(1—f 2 ) rankG . 
And the Weyl group is trivial, as we have already used above. We have 

d 

2A(A) = | (Af + a(A))j , 

i =1 

where (Ac + a (A)); is the W-component of Ac + a(A) e 7L d . 

Let us start the proof. A key is a trivial observation that the hyper-Kahler quotient of 
M by T d is a single point at any level of the hyper-Kahler moment map. It is enough to 
check d — 1, then it is obvious that the solution (x,y) G C 2 of 

J xy = Cc, 

[ M 2 - \y \ 2 = Cm 

is unique up to U(l) for any <^c- Let us see this in the GIT picture. Let (c = 0, as it 
becomes trivial otherwise. A function on xy = 0, which has weight m £ Z with respect to 
the C*-action z ■ (x,y) = ( zx , z _1 y), is x m if m > 0, y~ m if m < 0, up to constant multiple. 
This function has weight \m\ with respect to the dilatation action t ■ (x,y) = (tx,ty). 
Therefore with respect to the C* x C*-action, the character of C [x,y]/(xy) is 

(5.2) 

This trivial observation can be applied to our situation by considering hyper-Kahler 
quotients of A ic by G v , which are nothing but hyper-Kahler quotients of M by T d . We 
have the complex moment map 

: A ic ~> (LieG v )* <E) C = 0c- 

We consider it as a set of functions defining the subvariety (/i G ) _1 (0). It is a complete 
intersection, and the space of sections on (/i G ) _1 (0) and that of A4 G differ by the factor 
1/(1 — t 2 ) rankG = p G (£; A). Now we consider (/i G ) _1 (0) is a quotient of the subvariety 
(/Ac ) _1 (0) in M, where is the complex moment map of the Traction on M. The 
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latter is given by x t y, — 0 (i — 1,..., d) in the standard coordinate system (x*, y ? ;)f =1 of 
M. A section of C\ F with weight A with respect G y is a function on xyj % = 0 with weight 
A f + a(A), hence is a monomial in either Xi or yi according to the sign of (Af + a(A))j. 
Now we deduce (5.1) by the same argument in (5.2). We replace z m by z A , t by t 2A A) 
respectively. 

As a byproduct of this proof of the monopole formula, we obtain a linear base of the 
coordinate ring C[A ic\- It is given by monomials in components of the moment map 
and 

(5.3) Yl ( x f x)i or y^ aWi ) 

i 

for each A e Z d ~ n . Here we take if a(A); > 0 and otherwise. 

Let us give an explicit presentation of the coordinate ring C[A 4 C \- It is an algebra over 
C[{jc]) H ie polynomial ring over 0c by the moment map : Aic —> 0c- Let us denote 
the element (5.3) by z x . (The element gives the corresponding character z x , and there is 
no fear of confusion.) Then C [Aic\ I s 0Aez d - n with multiplication 

zV = z^TTiYl^^), 

l 

where 7r: C[tc] —> C[@c] is the projection induced by the inclusion 0c C tc, £i is a standard 


linear coordinate function 

on t^., and 












'0 

if 

a(A)i 

X 

> 0, 









if 

a(A)j 

> 

o, 

a(y)i < 

o, 

a(A 

+ 

y)i 

> 

o 

di(X,n) = < 

a(A )i 

if 

a(A)j 

> 

o, 

Oi{n)i < 

o, 

a(A 

+ 

y)i 

< 

o 


-a(X)i 

if 

a(A)i 

< 

o, 

a(n)i > 

o, 

«(A 

+ 

h)i 

> 

0 


k cr (p.) * 

if 

a(A)i 

< 

o, 

a(y)i > 

o, 

a(A 

+ 

h)-i 

< 

0 


This is because the moment map is induced from n^ d on M, which is given by Xiyi. 
We write by x^ X ^ or y^ a( ' X>t , and identify ^ with x % y % . Then we get the above formula 
of di( A, fi) according to signs of a(A)j, a(p)j, a(A + /i)j. 

This formula is found by Dimofte and Hilburn [ HI 4], 

6. Topological twist 

In this section we discuss PDE’s relevant to the topologically twisted version of our 
gauge theories. For (G, M) = (U(1),HI), it is nothing but the Seiberg-Witten monopole 
equation [ ]. More general cases were discussed for example in [ Tau99, Pid04, Hay 13]. 

However their reductions to 2-dimension were not discussed before, as far as the author 
knows, except it is of course well known that the dimension reduction of the Seiberg-Witten 
equation is the (anti-)vortex equation. Therefore we will discuss equations from the scratch 
though our treatment is more or less a standard textbook material. 
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6(i). Generalized Seiberg-Witten equations. We consider only the case when the 4- 
ma nif old X is spin for simplicity. See Remark 6.2 for a framework close to the usual 
Seiberg-Witten equation for a spin c 4-manifold under an additional assumption on (G, M). 

Recall that M is a quaternionic representation of G. Therefore Sp(l), the group of 
unit quaternions, acts on M commuting with the G-action. Then M can be made a 
representation of G d = Spin(4) x G = Sp(l) x Sp(l) x G in two ways, by choosing Sp(l) x 
Sp(l) —> Sp(l) either the first or second projection. Let us distinguish two representations, 
and denote them by M + and M . We write Spin(4) = Sp(l) + x Sp(l)_ accordingly. 

We regard El as an Sp(l)+ x Sp(l)_-module by left and right multiplication. The quater¬ 
nion multiplication gives a G'-homomorphism El 0 M —>■ M + . Combining this with 
El 0 M + —y M _ given by (x, m) i—>• — xm , we have a Clifford module structure (over M) on 
M+©M-. 

We regard the hyper-Kahler moment map (see §2(i) for detail), as a map /j: M + —> 
0 0sp(l)+ by identifying sp(l) + with the space of imaginary quaternions. It is equivariant 
under G. 

Let X be an oriented Riemannian 4-manifold. We assume X is spin, as mentioned above. 
Let P —» X be a principal G-bundle. We have the associated G-principal bundle, denoted 
by P, by taking the fiber product with the double cover Pspin( 4 ) of the orthonormal frame 
bundle Pso( 4 ) of TX given by the spin structure. Let A be a connection on P. We extend 
it to a connection on P, combining with the Levi-Civita connection on the Ps P i n ( 4 )-part. 
We denote the extension also by A for brevity. We have induced covariant derivatives Va 
on?Xg M + and P Xg M~. Combining them with Clifford multiplication, we have Dirac 
operators : T(P x^ M ± ) ATfPXg M T ). 

Let <f> be a section of P Xg M + . Applying the hyper-Kahler moment map hberwise, we 
regard //(<!>) as a section of P x^ ({J0sp(l)+). Note that P x^sp(l)+ = Pso( 4 ) x so(4)5p(l)+ 
is the bundle A + of self-dual 2-forms. Hence /x($) is a section of A + 0 (P x^j). 

Now we define a generalized Seiberg-Witten equation by 


D+4> = 0 

n = m < 4 ) 


Here Pj" is the self-dual part of the curvature of A. More precisely it is of the connection 
on P, not of the extended one on P. 


Remark 6.2. Assume that G contains a central element acting by the multiplication of —1 
on M. This is true for (G, M) = (U(1),EI) corresponding to the usual Seiberg-Witten 
monopole equation. Then we can replace G by its quotient G' = Spin(4) x G/ ± (1,1). 
The example G = U(l) gives G' = Spin c (4). Then we consider a principal G'-bundle 
P' together with an isomorphism P'/G = Pso( 4 ) of SO(4) = Spin(4)/± 1-bundles. The 
generalized Seiberg-Witten equation still makes sense for a connection A on P' and a 
section $ of P' Xq, M + . Here A is supposed to induce the Levi-Civita connection on 
Pso( 4 ). In this framework, we do not need to assume X to be spin, an existence of a lift of 
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Pso( 4 ) to a G"-bundle P' is sufficient. This framework is used in the usual Seiberg-Witten 
equation. 

When M is of the form O ©r HI for a real representation O of G, we have a further 
generalization. An example is M = g ©r HI. The point is HI has two H-module structures 
by left and right multiplications. 

We regard HI as a representation of Spin(4) = Sp(l)+ x Sp(l)_ as above. We consider M 
as a hyper-Kahler manifold by the left multiplication. And the moment map is regarded 
as /i: M —» g <8) sp(l) + as above. We have three possibilities to make M = O ©r HI as 
an Sp(l) + © Sp(l)_-module. (They are possible topological twists in physics literature.) 
Let (g + ,g_) G Sp(l) + x Sp(l)_ and x G HI. Then the action is either (1) x i—> g + x, (2) 
x i —y g+xg^ 1 , or (3) x H > g+xgZ 1 • The case (1) is the same as the case studied above. In 
the cases (2), (3), it is better to replace G by SO(4) x G as (—1, —1) G Sp(l)+ x Sp(l)_ 
acts trivially on HI. 

The case (2) means that we regard HI with S + (8) S + = A 0 © A + . Then <f> is a section 
of (A 0 © A + ) ®(Px G O). Let us write it by C © B. The Dirac operator Dj{ is identified 
with d A © d* A via the isomorphism S + © S~ — A 1 , where S ± is the spinor bundle. Hence 
the generalized Seiberg-Witten equation is 

d A C + d\B = 0, 

F A — fi{C © B). 

This is the equation in [V\\ ] if M = g ©r HI. 

Now consider the case (3). Then HI is identified with S + ®S~ = A 1 . Hence $ is a section 
of A 1 ® (P x G O). The Dirac operator is identified with d* A © d A via S~ ® S~ = A 0 © A'. 
Therefore the equation is 

= 0 = d* A < E>, 

Ft = M(4). 

This is the equation in [ ] if M = g ©r HI. For M = g ©r HI, there is a one-parameter 

family of equations parametrized by t G P 1 . See [ vW07, (3.29)]. 


6(ii). Roles of Higgs branch. Suppose X is compact. We return back to the equation 

( 6 . 1 ). 

From the Weitzenbock formula for D A for a solution (A, $) of generalized Seiberg-Witten 
equations, we have 


(6.3) 


o = Ia|$| 2 + |V A $| 2 + ^|$| 2 + (F|$,$) 
= ^A|<I>| 2 + |V A $| 2 + ^|$| 2 + 2|/i(<F)| 2 , 


where s is the scalar curvature. For the ordinary Seiberg-Witten equation, we have 
|/^( < F)| i = |$| 4 /4. Then considering a point where |$| 2 takes a maximum, one concludes 
sup |$| 2 < sup Y max(-s, 0). (See e.g., [Mor96, Cor. 5.2.2].) 
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The same argument works if there exists a constant C such that |0| 4 < C|/x(0)| 2 for 
(j) G M. However this is not true in general. If there is nonzero 0 such that /i(0) = 0, the 
inequality is not true. In fact, it is easy to check the converse. If /x(0) = 0 implies 0 = 0, 
the inequality holds. Thus we have proved 

• If the Higgs branch Xi H is {0}, $ is bounded for a solution of (6.1). 

The author learned this assertion during Witten’s lecture at the Isaac Newton Institute 
for Mathematical Sciences in 1996, 18 years ago. 

If M.h {0}, we should study behavior of a sequence of solutions (Aj, <3>j) with |<hj| —> 
oo. We consider the normalized solution = < f > j/|| < 3>j|| i 2. From (6.3) we derive a bound on 
sup|i>j|, || Va^iIIl 2 , and also f x |/z($j )| 2 < C|| < hj||£2 for a constant C. Thus converges 
weakly in W 1,2 and strongly in L p for any p > 0. Moreover converges to 0, i.e., the 

limit $oo takes values in ^ _1 (0). Thus the Higgs branch Xi H naturally shows up. 

Further analyses are given in e.g., [Taul3] for a special case (G,M) = (SU(2),g ©HI). 
See also [ IW15] for a special case (G, M) = (U(l),lHI Ar ) and dimension 3. 


6(iii). Kahler surfaces. Let us consider the case X is a compact Kahler surface, following 
[Wit9 1. §4], [Mor96, Chapter 7], 

The bundle A + of self-dual 2-forms decomposes as A°o; © A 0,2 , where c o is the Kahler 
form. It induces a decomposition = ( F x ) 1! 1 © F^ 2 . The moment map /^(d*) also 
decomposes as /%($) © /ti c ($). 

We replace G by its subgroup G = U(l) x Sp(l)_ x G. The principal bundle Ps P m(4) has 
the reduction Tu(i)xSp(i)_ to U(l) x Sp(l)_-bundle. We have a square root K of the 
anti-canonical bundle K x x , associated with the standard representation of U(l). On the 
other hand, the bundle associated with the vector representation of Sp(l)_ is A 0,1 © K x ' 2 . 
The bundle P likewise has a reduction P to a G-bundle. It is also the fiber product of P 
and Tu(i)xSp(i)_ over X. The connection A on P extends to a connection on P as before, 
by combined with the Levi-Civita connection. 

Since the multiplication of i on M commutes the action of U(l) x G, we can view M + 
as a complex representation of G. Hence P x^ M + is a complex vector bundle. A direct 
calculation shows that /x R (zd>) = /u R (<F), /t c (AI>) = — ^ C (<1>). 

We combine the complex structure i with the following which is a consequence of the 
Weitzenbock formula: 

(6.4) [ |D+4| 2 + i|F+- M (4)| 2 = / |V. 4 4>| 2 + W+| 2 + K(4)| 2 + 114| 2 . 

J X J X 

The right hand side is unchanged when we replace $ by zdL On the other hand, ( A , <E>) 
is a solution of the generalized Seiberg-Witten equation if and only if the left hand side 
vanishes. Hence ( A , <F) is a solution if and only if both (A, <F) and (A, *$) are solutions, 
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that is 


(6.5) 


0+4 = 0 = Dim, 
F°/ = 0 = M4), 
(#t) M = 


In particular, P XqGc has a structure of a holomorphic principal bundle induced from A. 
Then Px G M and M + = 0 (Px G M) are holomorphic vector bundles. 

We claim that $ satisfies = 0 = if and only if $, viewed as a section of 

(P Xg M + )*, is holomorphic. It is enough to check the assertion at each point p, hence 
we take a holomorphic coordinate system (zi,z 2 ) around p. We write Z\ — X\ + iyi, 
z 2 = x 2 + iy 2 . We assume tangent vectors d/dxi, d/dyi, d/dx 2 , d/dy 2 correspond to 1, i, 
j, k respectively under the isomorphism T p X = H. Then 




a=l 


_d_ 

OXn 


V a $ 

dx a 


_d_ 

dye 


V_ 8 _$ 

dVct 


= -V^_<F + zV^_<h + + kX7_a_$ 

dxi dyi dx2 dy2 

= -2 + 2 X_g_$j. 

dzi d%2 

If we replace $ by i<&, the first term is multiplied by i, and the second term by — i. Therefore 
the assertion follows. 


Theorem 6.6. A solution of the generalized Seiberg-Witten equation (6.1) on a compact 
Kahler surface X consists of a holomorphic Gc-bundle 7, a holomorphic section <F of 
K\[ 2 <g> (CP x g c M) satisfying 

Mc($) = 0 , 

(P +) 1 - 1 = 


The equation (P 4) 1 ’ 1 = /x R ($)o; corresponds to the stability condition by the Hitchin- 
Kobayashi correspondence. Therefore the moduli space of solutions has an algebro-geometric 
description. 


Remark 6.7. For the original Seiberg-Witten equation for (G, M) = (U(1),HI), /z c ($) = 0 
can be written as a/3 = 0 where $ = a + j/3. Therefore we have either a = 0 or /3 = 0. 
We do not have such a further reduction in general. 

6 (iv). Dimension reduction. Let us suppose the spin 4-manifold X is R x Y for a spin 
3-manifold Y. The 3-dimensional generalized Seiberg-Witten equation is an M-invariant 
solution of (6.1). We regard so (3) = sp(l) as the space of imaginary quaternions, and we 
have the Clifford multiplication so(3)®M — » M. We take a principal G-bundle P —> Y and 

consider the associated G-bundle P with G d =' Spin(3) x G, as in the 4-dimensional case. 
The 3-dimensional generalized Seiberg-Witten equation is a system of partial differential 
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equations for a connection A on P and a section $ of P Xq M: 


( 6 . 8 ) 


D a $ = 0 , 

* Fa — - r ($)- 


Here /i(<I>) is a section of P Xq (g ® so(3)), and considered as a P Xg g. valued 1-form. 
The Hodge star * send 2-forms to 1-forms in 3-dimension, the lower equation makes sense. 

The same remark on the bound on $ in 6 (ii) applies also in the 3-dimensional case: We 
need to understand the behavior of solutions (Aj, <3>j) with |<f>j| —x oo in order to define 
invariants rigorously. 

For the Floer homology group, we consider the equation 

s 4 = - BA 

jA=-*F a -h( J>) 

on Rxh This is the gradient flow equation for a generalized Chern-Simons functional 
given by 

S(A,$)= [ CS(A) + <F) dvol. 

Jy 2 

Here CS is the (normalized) Chern-Simons functional satisfying 


d 

dt 


X csm= X ( ^ aFj * )= 



*F At ) dvol. 


In the framework of a topological quantum field theory in 3d, it is natural to expect that 
one should consider the generalized Seiberg-Witten equation (6.8) for a 3-manifold Y with 
cylindrical ends. As ‘boundary conditions’, one should look at the translation invariant 
solutions of (6.8) over Y — R x C for a 2-dimensional Riemann manifold C [Don99, §2], 
Since C is a Kahler manifold, we can apply the argument in §6(iii) to reduce the equation 
to 

A defines a holomorphic GV-bundle CP on C, 

$ is a holomorphic section of K ] J' 2 <g> (CP x q c M), 

6,9 = o, 

* Fa — £%(*=&); 

where * is the Hodge star operator in 2-dimension. The first condition is automatic as C 
is complex dimension 1, and hence F^ 2 is automatically 0. 

This is the equation for the gauge nonlinear cr-model whose target is the Higgs branch 
M///G = n~\0)/G. 
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7. Motivic Donaldson-Thomas type invariants 

From the (2 + l)-dimensional TQFT framework explained in §l(v), the Coulomb branch 
M.c of Hyp(M fff- G ) is expected to be related to the cohomology of moduli spaces of 
solutions of (6.9). This is a starting point, and must be modified as we have explained in 
Introduction. We should get a reasonable answer for (G,M) = (SU(2),0) and we should 
recover the monopole formula (1.2) for good or ugly theories. 

Our proposal is as in §l(ix). Besides reproducing (1.2), these modifications are natural 
in view of recent study of Donaldson-Thomas (DT) invariants, as we will explain below. 
Recall that DT invariants were introduced as complex analog of Casson invariants [DT98]. 
As we can use algebro-geometric techniques to handle singularities in moduli spaces, DT 
invariants are easier to handle than the original Casson invariant in a sense. Therefore it 
should be reasonable for us to model the theory of DT invariants. 

7(i). Holomorphic Chern-Simons functional. Let C be a compact Riemann surface. 
We choose and fix a spin structure, i.e., the square root Kfi of the canonical bundle Kq. 
We also fix a (C 00 ) principal Ge-bundle P with a fixed reference partial connection d. (See 
Remark 7.1(b) below, for a correct formulation.) A field consists of a pair 

d + A : a partial connection on P. So A is a C^-section of A 0,1 <S> (P Xg c g c ). 

<f> : a C'°°-section of K\- 2 ®[Py. Gc M). 

Let T be the space of all fields. There is a gauge symmetry, i.e., the group Gc{P) of all 
(complex) gauge transformations of P natural acts on the space T. 

Our notation is slightly different from one in (6.9). We replace $ by $ for brevity, and 
A is a partial connection instead of a connection. 

Remarks 7.1. (a) When we have a flavor symmetry Gp, we slightly change the setting as 
follows. Recall we have an exact sequence 1 —> G —> G —> Gp —>■ 1 of groups. Choose a 
principal (GV)c-bundle Pp with a partial connection dp , and also a G-bundle P with a 
partial connection d . Then T consists of 

d + A : a partial connection on P. Moreover, we assume that the induced con¬ 
nection on P F is equal to dp. 

<3> : a C^-section of K\l 2 ® (P Xg c M). 

Since the notation is cumbersome, we will restrict ourselves to the case without flavor 
symmetry except we occasionally point out when a crucial difference arises. 

(b) A C°° principal Gc-bundle is classified by its ‘degree’, an element in tti(G). This is 
compatible with what is explained in §4(iii)(c). In particular, we need to take sum over all 
degrees to define the Coulomb branch. Since the sum occurs at the final stage, and is not 
relevant to most of calculations, we will not mention this point from now. 

We define an analog of the holomorphic Chern-Simons functional by 

(7.2) CS(A, $) = l f u c ({d + A)$ A $), 

2 Jc 
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where o;c( A ) is the tensor product of the exterior product and the complex symplectic 
form cue on M. Since (<9 yA)< 3> is a C'°°-section of /\ 0,1 ® K^ 2 (gi(Fx Gc M), u; c ((<9yA)<I>A$) 
is a C^-section of /\ 0,1 $3 Kq = A 1 ’ 1 - Its integral is well-defined. This is invariant under 
the gauge symmetry Qr{P). 

When M is a cotangent type, we can slightly generalize the construction. Let us choose 
M\, M 2 be two line bundles over C such that M\ ® M 2 = Kc■ We modify $ as 

<E>i, $2 : C'°°-sections of Mi <g) (P x Gc N) and M 2 ® (P x Gc N*) respectively. 

Then 

(7.3) CS(A $ 1; $ 2 ) = f ((5 Y A)^, $ 2 >. 

Jc 

It is a complex valued function on T. 


Remark 7.4. In this expression, analogy with the holomorphic Chern-Simons functional 
over a Calabi-Yau 3-fold X is clear. We consider a kind of dimension reduction from X to 
C, and connection forms in the reduced direction are changed to sections $ 1 , $ 2 . 


Let us consider critical points of CS. We take a variation in the direction ( A , <L): 

dCS {A ^(A, 6 ) 

— — / cuc((cl Y yl)*!? A $) -f- cJ(q((( 9 Y yl)$ A 4?) Y ouq^AQ A $) 

2 Jc 

= / cuc((cl Y A d*) Y (A A /x G ( < I ) )). 

Jc 

Therefore ( A , $) is a critical point of CS if and only if the following two equations are 
satisfied: 


(7.5) 


(9y4)$ = 0, 
Mc($) = 0 . 


1 /o 

The first equation means that is a holomorphic section of Kj 0 (P Xg c M) when we 
regard P as a holomorphic principal bundle by d Y A. Thus we have recovered the second 
and third equations in (6.9). 

When we have a flavor symmetry Gf, the partial connection <9p on Pp is fixed, and we 
only arrow A on P to vary in the ‘G-direction’. The Euler-Lagrange equation remains the 
same, if we understand d Y A involves also Of- 


Remark 7.6. When M is of an affine quiver type in §2(iv), the solution of (7.5) is analogous 
to quiver sheaves considered by Szendroi [SzeOS] . When we further assume M is of Jordan 
quiver type, it is also analogous to ADHM sheaves considered by Diaconescu [Dial2b]. 
In these cases, hyper-Kahler quotients parametrize instantons on C 2 /T (and its resolution 
and deformation) via the ADHM description [ vN90], where T is a finite subgroup of SU(2) 
corresponding to the quiver. We apply the ADHM description fiberwise, critical points give 
framed sheaves on a compactihed C 2 /T-bundle over C. We do not have the corresponding 
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3-folcl for general (G,M), but it is philosophically helpful to keep the 3-fold picture in 
mind. 

Moreover, one could probably consider analog of Gromov-Witten invariants, namely 
invariants of quasimaps by Ciocan-Fontanine, Kim, and Maulik, again imposing a stability 
condition [ ]. The twist by Kj is not included, and they are analog of usual 

Gromov-Witten invariants for stable maps from P 1 to //^ 1 (0)/Gc. 

Note that there are two bigger differences between our moduli and one in [3ze08, Dial2b, 
]: we assume P to be a genuine G-bundle (or a vector bundle for G = GL(iV)), 
while it is just a coherent sheaf in [SzeOS, Dial21>] and the base curve C is not fixed in 
[CFKM14], And a stability condition in [Sze08, Dial2b, CFKM14] is not imposed here. 

As is clear from Diaconescu and his collaborators [Dial2a, CDP12], a change of stability 
conditions is worth studying, but we cannot impose the stability condition in order to 
reproduce the monopole formula, as is clear from the special case (G, M) = (SU(2),0). 

If the space Z(g^) = {£c € 0c I Ad*(£c) = Cc f° r an y 9 £ G} is nontrivial, we could 
perturb CS by choosing a section (c °f Kc <H> Z(qq): 

CS^(A, $) = f ±u>c{{d + A)<F A <F) - (A A Cc). 

J C 

The Euler-Lagrange equation is 

(d + A)$ = 0, 

= Ce¬ 
lt is not clear whether this generalization is useful or not, but we will study the toy model 
case G = C in §9(ii). 

7 (ii). The main proposal. Our main proposal is that the coordinate ring of the Coulomb 
branch M.c of Hyp(M) -fjj-G is the equivariant cohomology (with compact support) of the 
vanishing cycle associated with CS: 

(7.7) C[Mc) = H* c +dirn-F-<hmg c ( p ) (sp ace 0 f solutions of (7.5), ^cs(Cf))*, 

up to certain degree shift explained later. Since T is infinite-dimensional, it is not clear 
whether the conventional definition of the vanishing cycle functor iff can be applied to our 
situation. We expect that it is possible to justify the definition, as the theory for usual 
complex Chern-Simons functional for connections on a compact Calabi-Yau 3-fold has 
been developed Joyce and his collaborators (see [ I F 15, BBB ] and the references 
therein). 

Since both dim P. dim Qc{P) are infinite (and its difference also as we will see below), 
we need to justify the meaning of the cohomological degree in (7.7). This will be done 
below by applying results for iff proved in a finite dimensional setting formally to our 
situation. Then cohomological degree in (7.7) will match with the grading on the left hand 
side coming from the U(l)-action, appeared in the monopole formula. We hope that this 
argument can be rigorously justified when the definition of the vanishing cycle functor is 
settled. 
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However, most importantly, it is not clear at this moment how to define multiplication 
in (7.7). In the subsequent paper [3FN16a], we replace P 1 by the formal punctured disc 
D x = SpecC((z)), mimicking the affine Grassmannian. We also replace the cohomology 
group by the homology group. Then we can define multiplication by the convolution. If 
M = 0, there is a natural pairing between the cohomology of the space for P 1 (thick affine 
Grassmannian) and homology of the affine Grassmannian (cf. [Gin95, Ch.6]). It is not 
clear whether this remains true in general at this moment. The reason which we take the 
dual of the cohomology group in (7.7) comes only from the comparison with the M = 0 
case, and hence it is not strong in our current understanding. 

Next we should have a Poisson structure in (7.7) from the complex symplectic form 
of Me- Ih [ 7 N16a], we consider C*-equivariant cohomology where C* acts on D x by 

rotation. It will give us a noncommutative deformation of the multiplication, and hence a 
Poisson structure in the non-equivariant limit. We call it the quantized Coulomb branch. 

The equivariant cohomology group (7.7) is a module over Hg c ^( pt). Therefore we 
should have a morphism from A4 C to Spec(Hg c ^ p Jpt)). It should factor through Spec(/7£(pt)) 
where Qc(P) — > Gc is given by an evaluation at a point in C, as Ker(^ c (P) —>■ Gc) acts 
freely on T. Note that Hq( pt) = C[gc] G = is a polynomial ring, where f)c is a 

Cartan subalgebra of 0c- Since we have dimfjc = dim.Mc/2, it is natural to expect that 
Me is an integrable system. 

7(iii)- Motivic universal DT invariants. Since (7.7) is too much ambitious to study 
at this moment, we consider less ambitious one, which is the motivic universal Donaldson- 
Tliomas invariant defined by 

[crit(CS)] vir 

1 J [Gc {P)U ' 

This is expected to be obtained by taking weight polynomials with respect to the mixed 
Hodge structure given by the Hodge module version of (7.7). The definition of motivic 
DT invariants have been worked out for degree zero by [BBS13], and also by [BBD + 15, 
BBBBJ15] and the references therein for general cases. 

Since we do not study the multiplication on (7.7) in this paper, our primary interest will 
be the Poincare polynomial of (7.7) and its comparison with the monopole formula. For 
C = P 1 , we introduce a stratification on the space (7 Z introduced below) in question such 
that each stratum has vanishing odd degree cohomology groups. Therefore the Poincare 
polynomial is equal to the weight polynomial, hence (7.8) looses nothing for our purpose. 

Since both T. Qc{P) are infinite dimensional, one still needs to justify the above defini¬ 
tion, defined again via the vanishing cycle functor. Also note that [<?c(-P)]vir is the virtual 
motive of Qc{P). i.e., 

(7.9) (—L ^r dimGciP) [Gc(P)}, 

which must be interpreted appropriately. 

Maybe it is worthwhile to mention one more thing need to be justified at this stage. 
The moduli stack of holomorphic GV-bundles over C is of infinite type. For example, over 
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P 1 , it can be reduced to Tr-bundles, in other words, sums of line bundles, but we have no 
control for the degrees of those line bundles. Therefore we need to stratify crit(CS) by the 
Harder-Narashimhan filtration, and consider motivic invariants for strata, and then take 
their sum. The last sum is infinite, so we need to justify it. In practice, at least for P 1 , the 
‘good’ or ‘ugly’ assumption means that we only get positive powers of L for each stratum, 
and we consider it as a formal power series in L. 

Remark 7.10. We call the above the universal DT invariant, as no stability condition is 
imposed following [MMNS12]. It is a hope that usual DT invariants and wall-crossing 
formula could be understood from the universal one as in [ JNS12], In [ NS12], 

the authors considered representations of a quiver, hence arbitrary objects in an abelian 
category. However, we do not include coherent sheaves (for G = GL), it is not clear 
whether this is a reasonable hope or not. 

7(iv). Cutting. We suppose that M is of cotangent type, so M = N © N*. We consider 
the scalar multiplication on the factor N*. Then we have (t -v,t-w) = t(v, w ) for t G U(l). 
We have the induced C*-action on T given by 

(A <h) ^ (A,t- $). 

Then CS is of weight 1: 

CS (A,t-$) = tCS(A, $). 

Thus the result of Behrend-Bryan-Szendroi [ 3S13] can be applied (at least formally) to 

get 

[crit(CS)] vir = —(—iJ)-^ ([CS-^l)] - [CS-^O)]) . 

We introduce the reduced space of fields by 

^red d ={(9 + A$l)}=^ C ‘- 

We have a projection T —> T xc d induced from M — > N. Note that CS is linear in <f> 2 
once ( d + A, dq) is fixed. It is zero if (d + H)^! = 0, and nonzero otherwise. (See (7.3).) 
Therefore the argument by Morrison [Morl2] and Nagao [Nagll] can be (at least formally) 
applied to deduce 

[CS-^l)] - [CS-^O)] = — (—L^) 2dim ( fiber ) [7^,], 

where TZ is the locus {(5 + H)<3>i = 0} in P rc d. Hence 

(7.11) [crit(CS)] vir = (-L^) 2dim ( fiber )- dimJ [K]. 

Note that 1Z is the space of pairs of principal holomorphic G'c-bundles (P, d+A ) together 
with holomorphic sections of the associated bundle Mi © (P Xg c N). 

If we take Mi = Kc (and hence M 2 = Oc) and N = N* = gc, 1Z is the space of Higgs 
bundles, introduced first by Hitchin [Hit87]. 

The fiber in (7.11) is the space of <h 2 , i.e., all C°° sections of M 2 © (P x Gc N*). This 
is obviously infinite dimensional, and hence dim(fiber) must be interpreted appropriately. 
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Observe that dim P is also infinite dimensional, however. Note that we also have dim Q (P) 
in (7.9). We combine them ‘formally’ to get 

2 dim(fiber) — dim T + dim GAP) 

(7.12) _ 

= dim{<f> 2 } — dim{<f>i} + dimCTc(-P) — dim{<9 + A}. 


Next note that 


dim{<f)2} — dinijlh]} 

= dim C°°(Kc ® M* ®(Px Gc N*)) - dim C 0O (M 1 8(Px Gc N)) 

= dim C°° (A 0,1 ® M x »(Px Gc N)) - dim C°°(M 1 ® (P x Gc N)), 

where the second equality is true because ‘dim’ is the same for the dual space. Now this 
is safely replaced by a well-defined number thanks to Riemann-Roch: 

— ind(<9 on M x 8 (P x Gc N)) 

= - deg(Mi (g) (P x Gc N)) + rank(M x ®(Px Gc N))(^ - 1), 

where g is the genus of C. Let us write this number by — *(Pn)- Similarly we have 

dim GAP) ~ dim{<9 + A} = ind(<9 on P x Gc jj c ) 

= dim G c (l - g). 


Let us denote this by i(P). We have 


(7.13) 


[crit(CS)] vir = Ui(P)- if P N ) [R] 

[GAP)U 1 [GAP)}' 


This argument can be ‘lifted’ to the case of the cohomology group of vanishing cycles 
thanks to a recent result by Davison [Davl3, Th. A. 1]. Applying his result formally even 
though the fiber is infinite dimensional, we get 


rj-*+dim T —dim Qc 

n c,g c (P) 


{p, ^Cs(Cjr)) 


= H 
( 

= H 


*+dim T— dim Qc~ 2 dim(fiber) 

c,Gc(P) 


(77, C) 


*-i(P)+i(P N ) 

c,Qc(P) 


(77, C). 


7(v). Second projection. There is a second projection 77 — y A(P), where A(P) is the 
space of all (partial) connections. The fiber at d + A e A(P) is the space of holomorphic 
sections Ker(c7 + A on M x ® (P x Gc N)). This depends on the isomorphism class of <9 + A. 
Let us denote the isomorphism class by [CP]. If we write CP, it will be a representative of 
[CP], considered as a holomorphic principal Gc-bundle. Let 

h°(CP N ) d = dim P°(M] 0 (CP x Gc N)). 




40 


H. NAKAJIMA 


Then we have 


(7.14) 


(_Li)-*( p Nr)+h p ) 


m 

[<w(A] 


E <- Ll ) 


HP)-i(PN)+2h 0 (3’ N ) 


mnA(P)/Sc(P) 


[Sc (p) • g] 

[Sc (P)} 


-- E (-^ 

meA(P)/G c (P) 


i(p)-i(p N )+2h°(y N ) 


1 

[Aut(CP)] 


This is a well-defined object. 

Note that 

- i(p N ) + 2 h°(? N ) = h°(v N ) + h\y N ) 

where /^(CPn) = dim H l (M \® (tP©^N)), and Tn*, Tm are vector bundles associated with 
representations N*. M, and h° denotes the dimensions of their holomorphic sections. In 
particular, this combination is always nonnegative, and makes sense even when M is not 
necessarily cotangent type. Therefore it is reasonable to conjecture that this computation 
remains true even without assuming M is of cotangent type. 


8. Computation for C = P 1 

8(i). Parametrization of Gx-bundles. Let us assume C — P 1 . Then all holomorphic 
principal G'c-bundles can be reduced to Tc-bundles, and they are unique up to conjugation 
by the Weyl group. This is a result of Grothendieck [Gro57]. Note that line bundles are clas¬ 
sified by its degree on P 1 . Therefore isomorphism classes of Gc-bundles are parametrized 
by the coweight lattice Y of G modulo the Weyl group W, or dominant coweights in 
other words. A coweight is called a magnetic charge in the physics literature. Therefore a 
coweight is denoted by m in [ 7 A 1], but we denote a coweight by A following a standard 
notation in mathematics. 

Let us explain these more concretely. Suppose Gc = GL(iV). Then a Gc-bundle is 
nothing but a rank N vector bundle. It decomposes into a direct sum of line bundles 
Cpi(Ai) © ■■■ © (Dpi (Atv)• As the order of sum does not matter, we may assume Ai > 
• • • > A n- This is the dominance condition. We can make arbitrary A = (Ai,... , Ajv) to a 
dominance one by the action of the symmetric group, which is the Weyl group of GL(iV). 

Keep in mind that rank and degree of the vector bundle are 

rank = N, deg = A,-. 

The degree is called the topological charge, denoted by J(A) in [ 1Z14, (2.7)]. See (4.4). 

By our formula (7.14), the motivic invariant is the sum over all isomorphism classes, i.e., 
dominant coweights. 

When we have a flavor symmetry Gf, we add a (GV)c-bundle with a partial connection 
dp as additional data. Its isomorphism class is also parametrized by a dominant coweight 
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Xp. It is a background flux, i.e., it enters in the formula, but it is fixed, and we do not sum 
over X F . 

Let us note also that the calculation below is true in the level of Poincare polynomials, not 
only as motives. Considering isomorphism classes of G'c-bundles, we have a stratification 
of the space 1Z. Each stratum is a vector bundle over a £?c(-P)-orbit, and hence has no odd 
cohomology groups. Therefore the short exact sequences associated with the stratification 
splits, hence the cohomology group is isomorphic (as a graded vector space) with the sum 
of cohomology groups of strata. 

8(ii). Calculation of contributions from matters. Let B( N) be a base of N compat¬ 
ible with weight space decomposition. For b 6 B( N), let wt(fe) be its weight. Hence 

N = 0C6. 

b 

We denote the pairing between weights and coweights by ( , ). When a principal bundle 
CP is reduced to a Tc-bundle, the associated vector bundle decomposes as 

CPn = (J) CP x^ c Cb, 

b 

a sum of line bundles. If A is the coweight for CP, the degree of the line bundle is (A, wt(6)) + 
deg Mi. Then 

h°( CP N ) + h 1 ( CP N ) = ^ |(A,wt(6)) + degMi + 1|. 

b 

This is because 

dim H°(Opi(n)) + dim H l (Ofi{n)) — \n + 1|. 

For our standard choice M L = M 2 = O p i(— 1), we have 

|(A,wt(6))|, 

b 

which coincides with the second term in A(A) in (4.1) if we replace —by t. (Recall A(A) 
appears as t 2A A) in (1.2)). 

The calculation remains the same for M not necessarily cotangent type. 

8(iii). Calculation of automorphism groups (G<c = GL(N, C) case). Let us first 
suppose that Gc is GL(iV, C). Then CP is a usual vector bundle of rank N. We write 
A = (Ai > A 2 > ••• > Ajv), a non-increasing sequence of integers. We also write A = 
(... (— l) fc - 1 0 fc °l fcl • • •), which is a modification of the usual notation for partitions, i.e., — 1 
appears k_\ times, 0 appears k 0 times, etc in the sequence A. 

In this notation, we have 

dim End(CP) = k a kp dim Hom(Opi(a), Of ,i (/?)) 

a,[i 

= '^k a kp(P - a + 1). 

/?>a 
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For Aut(CP), we replace the block diagonal entries a = f3 by GL (k a ). Therefore 

(S-D [Aut^^WUll^L. 


The ratio [GL(/c)]/[End(C fc )] is easy to compute and well-known: 

[GL(fc)]/[End(C fc )] = (L fc - l)(L fc - L) ■ ■ ■ (L fc - L fc_1 )L~ fc2 
= (-l) fc (l - L)(l - L 2 ) • • • (1 - L^L - *^ 2 . 


Thus 


where 


(-L^( p ) 

[Aut(T)] 


a 


1 

(1 — L)(l — L 2 ) •••(! — L fca ) ’ 


d = i(P) — 2 dim End(CP) + k a (k a + 1) 

= - 2 ^ k a k^{/3 - a) - 2 ^ k a k p + ^ k a kg + ^ k a (k a + 1) 

13>a /3>a: a,(3 

= - 2 k a kp(/3 - a) + k a . 

(3>a 


The first term is 

-2]T(A,-A,), 

i<j 

which is equal to the first term in A(A) in (4.1), again if we replace —La by t. 

The term 

TT_I_ 

AA (1 — L)(l — L 2 ) ■••(! — h ka ) 

is Pu(A)(^; A) if we put L = t 2 . 

Therefore up to the factor (—l)^ fc “(—it coincides with (1.2). Note that J2k a 
is N = rankG, and hence is independent of A. 

Recall that the rank of the group is dime M-c (§4(iii) (3)) . Therefore 




n jh-dim uMc [ cr it(CS)] vir 
( _) [Gc(P)Ur 


The virtual motive is shifted already by — dim/2, hence Hc.m( t) is shifted by — dim in 
total. This shift is unavoidable, as the monopole formula starts with 1 which corresponds 
to constant functions on A4c- This convention is not taken in the cohomology side. 
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8(iv). Calculation of automorphism groups (general case). Let us turn to a general 
case. The relevant calculation was done in [ SD07, §7]. We reproduce it with shortcuts of 
a few arguments for the sake of the reader. 

As in [ , Def. 2.4], we add the torsor relation to our motivic ring: [CP] = [X][G] for 

a principal G bundle CP over a variety X. 

Let P, U, L be the parabolic subgroup, its unipotent radical and its Levi quotient 
associated with the dominant coweight A. (L = Stabc(A) in the previous notation.) Since 
the underlying C°° principal G'c-bundle P will not occur any more except through the 
formula i(P) (which is dim Gr in our case), there is no fear of confusion. Let u be the Lie 
algebra of U, and CP U be the associated vector bundle. Then generalizing the computation 
for G = GL(N, C), we have 

Lemma 8.2 ([ , Prop. 5.2]). We have Aut(CP) = L ix Lf°(P 1 ,CP u ). 

The proof is not difficult. It could be an exercise for the reader. 

Since CP is now a T-bundle, we have 

A= © A., 

aeA+ 

(A,a)>0 

where CP gQ is the line bundle associated with the root subspace g a . Its degree is (A, cc). 
Hence 

dim if 0 (P 1 , CP U ) = ^ «A, a) + 1). 

aeA+ 

(A,a)>0 

Therefore 

IgTUM _, 2(j, P) 
p] 

where p is the half sum of positive roots. Note 2 (A , p) = EaeA+(^ “)■ Since we take A 
dominant, this is the first term (up to sign) of A(A) in (4.1). 

We substitute this into Lemma 8.2. Since LU = P, 

[Aut(?)] = L 2 M[P] = L 2 <^> AL, 

where G/P is the partial flag variety. (We implicit used the torsor relation. See [15D07, 
p.637] for detail.) 

The partial flag variety G/P has the Bruhat decomposition. Therefore we only need to 
compute ordinary cohomology. To connect with the ring of invariant polynomials, we use 
an isomorphism of equivariant cohomology groups: 

H’ a (G/P) = H' L ( pt). 

The spectral sequence relating equivariant and ordinary cohomology groups collapses for 
G/P (as there is no odd degree cohomology). Hence Hq(G/P ) = H*{G/P ) © Hq{ pt). 
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Thus we get 

ir/P] = [SMI = p c(-iAa) 

[-Hc(pt)] Pg(-L^; 0) 

From the special case P — B of this computation, we have 

[B] = [G\Pg(—IP'i 0)(1 — L) rankG , 

as [TTf(pt)] = (1 — L)” rankG '. On the other hand, B is the product of T and its unipotent 
radical. Hence [B\ = (L — 1) rank G [L] 5( dim G i~ rank c) . yy e g e {- 

[G]P G (-L3;0) = (-l) rankG [L] 5 (dimG C —rankG)_ 


Now we combine all these computation to get 


(-L5)^ p ) 

[Aut(CP)] 


(_ 1 )«nkG(_Ll)—4(A,p>+rankGp G ^_ L | ; 


This coincides with the contribution of terms not involving M in the monopole formula up 
to the factor (—1) rank G (—L^) rank c . 


9. Computation for C = C 

9(i). Motivic universal DT invariant for 0c x M. Since we assume C is compact, we 
cannot apply the argument in §7 to C = C. However we further consider the dimension 
reduction in the C-direction to a point as in Remark 7.4. In practice, it means that we 
replace d + A by an element £ in the Lie algebra 0c- We thus arrive at 

CS(£,$) = ^ c (£d>, *) = <£, 

This is a function on a finite dimensional space J = gc x M, and all computation in §7 
become rigorous. We repeat our assertions in this setting: 

(1) (£, <F) is a critical point of CS if and only if 

= 0, a*c($) = 0. 

(2) The motivic universal DT invariant is defined by 

[crit(CS)] vir 

[Ccjvir 

(3) Assuming M is of cotangent type (i.e., M = N © N*), we use a C*-action given by 

(£, $i, $ 2 ) (£, $ 1 , t$ 2 ) to get 

[crit(CS)] vir _ [77] 

[Cc]vir [G*c] ’ 

where 77 is the locus £$1 = 0 in .F red = {(£, (Ir) 6g c x N}. 

Note that (7.12) vanishes in this setting. 
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(4) We further have 

,, [crit(CS)]vir _ J^]_ _ ^ L dim(KergonN) 

[G c ] vir [G c ] { ^ /Gc [Stab Gc (0] 

All these are rigorous now. 

9(ii). Another C*-action. It is interesting to note that we have another C*-action (£, <f>) i-> 
(££, $), which makes sense without the cotangent type condition. It is of weight 1, even 
when we perturb /x c by Cc G Z(q^). The cutting for this action was used by Mozgovoy 
[Mozll] for (G, M) of quiver type. 

Then the application of results on the cutting implies as in §7(iv) 

(q 9 \ [crit(CS)] v jr _ , l\-(dimM-2dimfl C ) [^Q 1 (Cc)] 

1 J [Gdvir [G C ] ' 

Note that dimM — 2 dim 0r is the expected dimension of Me (Cc)/Gc- 

Let us again suppose M is of cotangent type and consider the restriction of the projection 
Mc^Cc) -A N* given by ($i,$ 2 ) •-> $ 2 - When M is of quiver type, it was studied by 
Crawley-Boevey and his collaborators [CBH98, CB01, CBVdB04], The argument can be 
modified to our setting as follows. 

Let us fix $ 2 € N* and consider an exact sequence 

0 —> Lie(Stab Gc ($ 2 )) - 1 0c ~t N*, 

where the second arrow is the inclusion and the third arrow is the action of Gc on N* : 
£ £$ 2 . We consider the transpose 

N —t 0 c Lie(Stab Gc ($ 2 ))* —> 0 . 

From the definition of the moment map, the first arrow is given by $1 1 —y 1 , $ 2 ). Let 

K$ 2 denote the kernel of N —)■ 0c- 

If we have a solution of $ 2 ) = Cc then £c is in the image of the first map. Hence we 
must have t( Cc) = 0 by the exact sequence. Moreover, if t (Cc) = 0, the space of solutions is 
an affine space modeled by Kq, 2 . Therefore we are led to introduce the following condition: 

Definition 9.3. We say <f >2 is Cc -indecomposable if t{ Cc) = 0, in other words, 

(£, Cc) = 0 for any £ G Lie(Stab Gc (4>2)). 

If M is of quiver type (with W = 0) and Cc is generic, this is equivalent to that $2 is 
indecomposable in the usual sense. See the proof of [ JVdBO 1, Prop. 2.2.1], On the other 
hand, this imposes nothing if Cc = 0. 

This condition is invariant under GV-action. Let N* r ind be the constructive subset of 
N* consisting of Cc-indecomposable $ 2 - 
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Let us go back to (9.2). We have 


(9.4) 


( 


-L3 


(dim M —2 dim 0c) 


[/V(Cc)] 

[Gc] 


E 

[4> 2 ]eN* c _ ind /G c 


E 

[4> 2 ]eN* c _ ind /Gc 


jj^dim K$ 2 —dim N+dim 0c 

[Stab Gc ($ 2 )] 

[Lie(Stab Gc (4> 2 ))] 
[Stab Gc ($ 2 )] ’ 


where we have used the exact sequence to compute the alternating sum of the dimension. 

In the situation of [ VdB04], we replace Gc by its quotient Gc/C* as explained at 

the end of §2(iv). If (c is generic and hence 4> 2 is indecomposable in the usual sense, 
Stab GlC /c*( < I ) 2 ) and its Lie algebra is isomorphic by the exponential. Then each term in the 
summand is 1. Hence the sum is the motif of the space of indecomposable modules. This 
is one of crucial steps in their proof of Kac’s conjecture for indivisible dimension vectors. 
See [CBVdB04, Prop. 2.2.1], 

Suppose (c — 0, hence <f> 2 is an arbitrary element. When ( G , M) is of quiver type with 
W = 0, the right hand side of (9.4) was computed in [ ]. Let us introduce variables 

Xi for each i G Q 0 and denote X T by x v for v = (vi) iG Q 0 . Summing up to the motivic 
Donaldson-Thomas invariants for all dimension vectors, the generating function is given 
by 


(9.5) 


y- [crit(CS)] vir 

V [G C ]vir 


exp 



^ v a v (L d )q: dv \ 
d( 1 - L- d ) ) 


) 


where a v (q) is the Kac polynomial counting the number of absolute indecomposable rep¬ 
resentations of dimension vector v of the finite field ¥ q . See [Mozll, Th. 1.1], 

Note that Kac polynomials have combinatorial expressions very similar to the monopole 
formula due to Kac-Stanley (see [I\ac83, p.90] and also [HuaOO] for a detail of the proof). 


Appendix A. Instantons for classical groups 


We give further examples of hyper-Kahler quotients arising as instanton moduli spaces 
for SO / Sp groups on M 4 with various equivariant structures. 


A(i). The case of M 4 . It is well-known that the ADHM description of SU(AQ-instantons 
on M 4 can be modified for SO / Sp groups. 

For SO(iV) Auinstantons, we take the vector representation W of SO(iV), the vector 
representation V of Sp(fc) and set 


(A.l) 


M = {(£ ls B 2 ,a, b ) e Hom(K, V)® 2 ® Hom(W, V) ® Hom(V, W) 
(B a v,v') = (v,B a v') for v,v' G V, a — 1,2,1 
(aw, v ) = (w, bv) for v G V, w E W [ 


where ( , ) is either the symplectic form or orthogonal form on V or W respectively. Then 
the framed instanton moduli space is the hyper-Kahler quotient /j," 1 (0)/ Sp (k). 
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For Sp (N) fc-instantons, we replace W by the vector representation of Sp(iV), V by the 
vector representation of O (k). It should be warned that the group is not SO (k). 

A(ii). Nilpotent orbits and Slodowy slices for classical groups. Let us generalize 
discussions in §2(v) to classical groups. This straightforward generalization was mentioned 
[Nak94, Remark 8.5(4)] for the SU(2)-equivariant case, but not explicitly written down 
before. 

Suppose (Bi, B 2 ,a,b) in (A.l) corresponds to an SU(2)-equivariant instanton. Then V, 
W are representations of SU(2), and (Bi, B 2 ), a, b are SU(2)-linear. Here we mean the pair 
(Bi, B 2 ) is SU(2)-equivariant, when it is considered as a homomorphism in Hom(V, V®p 2 ), 
where p 2 is the vector representation of SU(2). 

Let us decompose V, W as Wi®pi as in §2(v). Since p, L has a symplectic or 

orthogonal form according to the parity of z, we have either symmetric or orthogonal forms 
on Vi , W t . For example, for SO (A)-instantons, linear maps B i, B 2 , a, b can be put into a 
graph (see Figure 6) as before. Groups Sp(^), 0(u>i), etc in circles or squares indicate, we 



Figure 6. Intersection of a nilpotent orbit and a Slodowy slice 

have a symmetric or an orthogonal form on Vy, Wy, etc. (Here ly = dim V), Wi = dim Hq.) 
The space M consists of linear maps, both directions for each edges, satisfying the transpose 
condition like in (A.l). And the group G, by which we take the hyper-Kahler quotient, 
is the product of groups in circles. It is not relevant here whether we should put either 
O(iUi),... or SO(ini),..., as we only need orthogonal forms. However, it is matter that for 
0 ( 112 ),. • •, as G is the product of groups in circles. As a special case w 2 = W 3 = ■ ■ ■ = 0, we 
recover Kraft-Procesi’s description of classical nilpotent orbits [I ]. This special case 
also appeared later in [ vS96]. 

A nilpotent orbit 0\ in SO corresponds to an even partition A. It corresponds to that 
dim M/even is even, and is compatible with that IFeven has a symplectic form. Similarly O fl 
corresponds to an even partition as dim VAdd is even. 

A(iii). Affine Grassmannian and S' 1 -equivariant instantons. The discussion in the 
previous subsection can be modified to the case of S' 1 -equivariant instantons on M 4 . We 
have a different quiver as the dual of pi is p-i for irreducible representations of S 1 . 

For an SO(r)-instanton, the corresponding quiver is Figure 7. From the isomorphisms 
V = V*, W = W* given by the symplectic and orthogonal forms, we have V) = V* i} 
Wi = HA*- Therefore we do not need to consider V), W t for i < 0. The moment map takes 
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Figure 7. W^ A for G = SO(r) 


value in R 3 © (sp(u 0 /2) © u(i>i) © u(u 2 ) ©■■•)• The u(i>i) © u(u 2 ) ©-component is as for 

ordinary quiver gauge theories. The sp (vq /2)-component is given by 

— B^ 0 Bq x + i 0 io, 

where B 0t i: V\ —> Vo, B i j0 : Vo Vi, io'- Wo —* Vo- And V is defined by (-Bi )0 vo, u_i) = 
(vo, B± 0 V-i), by using the symplectic pairing ( , ) between V\ and Vli, Vo and itself, etc. 
Thus (G, M) for SU(2)-equivariant and S' 1 -equivariant instantons are different for classical 
groups. 

A(iv). The case of R 4 /T. Let T be a hnite subgroup of SU(2). Consider a T-equivariant 
G-instanton on R 4 . The case G = U(T) was explained in §3(ii). So we explain how it can 
be modified to the case when G is SO / Sp. 

As above, the isomorphisms V = V*, W = W* induces V = V**, Wi = W**, where i* is 
determined by the McKay correspondence as p* = p^. It fixes the trivial representation 
Po, and hence induces a diagram involution on the finite Dynkin diagram. It is the same 
diagram involution given by the longest element u> 0 of the Weyl group as — w 0 (cu) = a i*, 
where oii is the simple root corresponding to the vertex i. In the labeling in [Kac90, Ch. 4], 
it is given by i* = £ — i + 1 for type 1* = 5, 2* = 4, 3* = 3, 4* = 2, 5* = 1, 6* = 6 
respectively. For type Di with odd £, it is given by 

— 1 if i — £, 

i* — < £ if i = i — 1, 

If otherwise, 

It is the identity for other types. 

When i* = i, we determine whether p t = p* is given by a symplectic or orthogonal form 
as follows: The trivial representation, assigned to i — 0, is orthogonal. For type Ag with 
odd i with i — (£ + l)/2, it is orthogonal. For other types, p io for the vertex i 0 adjacent to 
the vertex i — 0 in the affine Dynkin diagram is the representation given by the inclusion 
T C SU(2). Therefore pi 0 = p* Q is symplectic. If i is adjacent to io and i* = i, then p t = p* 
is orthogonal. If j is adjacent to such an i with j* = j , then pj = p* is symplectic, and so 
on. From this rule, we determine either Vi = V*, W t © W* is symplectic or orthogonal. 
We impose constraint on linear maps B , a, b between Vi, Wi s induced from (A.l) either 
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Examples discussed in §A are not of cotangent type in general. We also remark that G 
is not connected if it contains O (N) as a factor. A criterion on the complete intersection 
property in §2(iii) is not known in general. See [Chol r ] for special cases. 

Appendix B. Hilbert series of symmetric products 

We compute 

Hk(t,z)= Jv l Al lp U ( fc )(t, A). 

A=(Ai>--->Afc) 

We formally set Ho(t,z) = 1 and consider the generating function over all k. Our goal is 
to show that 


(B.l) 


A d 


Y H k (t , z)A k = exp ( Y ~J Hl(yt<1 ' ^ 


k =0 


v d= 1 


The Erst term Hi(t,z) = (1 — t 2N )/(l — t 2 )(l — t N z)( 1 — t N z _1 ) is the Hilbert series of 
C 2 /(Z/./VZ), and the above is its plethystic exponential. Then it is the Hilbert series of 
symmetric powers. 

Let us first note that there is a one-to-one correspondence between dominant coweights 
A = (Ai > • • • > Afc) and (fco, An, • • • ; m) G Z > 0 x Z such that 

(a) k 0 7 ^ 0 . 

(b) ko + k\ + ■ ■ ■ — k. In particular, there are only finitely many nonzero k a . 

The correspondence is given by m — X k and k a = #{i | A* — X k = a}. This is an extension 
of usual two types of presentations of partitions. 

We write (f; t) k = (1 — f)(l — f 2 ) ■ • • (1 — t k ) as usual. Under the bijection we have 

Y x i = Y ka ( m+a ^ ia » i = Y k c‘\ m + a \^ 

a a 

l 


Pv(k){t,X) = J| 




Hence 


OO OO 


e^.(m)v=i + e e n 


^N\m+a\k a z (rn+a)k a A k a 


k =0 


mEZ ko=l a=0 
k\ ,•••—0 




^En e 

mGZ a=0 k a = 0 for q/0 
ka = 1 for a = 0 


j-N | m+a | k a ^ (m+a)k a ^k Q 


+-,t 2 ) f 


By the g-binomial theorem (see e.g., [Mac95, Ch. I, §2, Ex. 4]) we have 

^N\m+a\k a z (m+a)k a A k a ^ 


E 

k a = 0 


+++ 


(t N \ m + a \zm+^A;t 2 ) 00 ' 1 
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where (a; t)^ = (1 — a)(l — at) ■ ■ ■. For a = 0, we subtract the first term k 0 = 0, which is 
1. Hence this is equal to 


(B.2) 


i+E n 


-A ( tN\m+a\ z m+o. A ; J-Jj (jJV|m+o| z m+oA; t 2 ) 


n 


mEZ \Q!—0 

We separate the sum to two parts m > 0 and m < 0. First suppose m > 0. Note that 
\m + a\ — m + a in this case. We change m + a to a and get 

oo / oo ^ oo ^ 

I n (t Na Z a A] £2) “ II (t Na Z a A] t 2 )^ 

m =0 \a=m v ' a=m +1 x 7 

We cannot take the sum o separately as each sum diverges. But it is possible once we 
subtract 1 from each term. Hence we get 


OO / OO 


( n (+Na z a\.f 2 ) 
m=0 \a=m ' 


e n 

ra=0 \o:=m+l 


(■ t Na z a A■ t 2 ) c 


n 


Ji(«"“z“A;i 2 ) c 


1 . 


The last —1 cancels with 1 in (B.2). 
Let us turn to m < 0. We have 


y ( TT_I_TT_I_ 

^ (^N\m+o\ z m+a.^ t 2^ 11 (jN\m+a\ z m+a ^ t 2 ) c 


m <0 \ol 
oo 


n 


oo / rri 

e n 


m —1 

n 


.1 (2)^ y±l (tw« z -»A;e)„ 11 A;i 2 ), 

We use the same trick as above. We subtract 1 from each term in the sum Y2m=i an d 
separate the sum. We get 


n 


n 


1 = 1 ((»«z«A;t 2 ) co \1_1 (t N «z~°A-,t 2 ) x 
Adding 1 and terms with m > 0, m < 0, we get 

~ 1 1 


- 1 


n 


n 


11 ( t N« z *A- t 2 )oo 11 {t Na z~ a A;t 2 ) 00 ' 

Now it is straightforward to check (B.l). 

Note Added in Proof 


After this paper was posted to ArXiv, several important progresses have been made. 
Bullimore et al. write a paper [ DG15], where it is argued from a physical intuition that 
C[Adc] is embedded into a localization of another C[Af c\ of the abelian gauge theory where 
the gauge group G is replaced by its maximal torus T. This embedding is rigorously given 
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in the definition of [ N16a]. At the same time, [BDG15] discusses Coulomb branches 

of quiver gauge theories of type ADE when // is not necessarily dominant. It clarifies a 
problem raised in §2(v). 

The sequel [ ; N16a], announced here, is written, and it is shown that the definition 

there reproduces many examples in quiver gauge theories [JJFNlhb]. Further examples 
have been studied in later papers by the author with collaborators. 
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